Harrell’s rms

See chapters 1 and 9 of Biostatistics for Biomedical Research.

Setup

require(Hmisc)
knitrSet(lang='markdown', h=4.5)
options(grType='plotly')
mu <- markupSpecs$html

R Packages

Install the packages Hmisc, rms, knitr, rmarkdown through RStudio.

Packages contain R code and may often require the installation of other packages. When I install Hmisc, I see that its dependency acepack is automatically installed:

Installing package into ‘/home/beckca/R/x86_64-pc-linux-gnu-library/3.1’
(as ‘lib’ is unspecified)
also installing the dependency ‘acepack’

trying URL 'http://debian.mc.vanderbilt.edu/R/CRAN/src/contrib/acepack_1.3-3.3.tar.gz'
Content type 'application/x-gzip' length 33590 bytes (32 Kb)
opened URL
==================================================
downloaded 32 Kb

trying URL 'http://debian.mc.vanderbilt.edu/R/CRAN/src/contrib/Hmisc_3.14-6.tar.gz'
Content type 'application/x-gzip' length 611348 bytes (597 Kb)
opened URL
==================================================
downloaded 597 Kb

* installing *source* package ‘acepack’ ...
.
.
.
* DONE (acepack)
* installing *source* package ‘Hmisc’ ...
.
.
.
* DONE (Hmisc)

Install and updating

Install through CRAN repositories

# install several packages
install.packages(c('Hmisc', 'rms', 'knitr', 'rmarkdown'))
# update all packages
update.packages(checkBuilt=TRUE, ask=FALSE)

Install through Github repositories

install.packages('devtools')
require(devtools)
install_github('harrelfe/rms')

Loading

Note that library is a bit of misnomer as R uses packages, not libraries. From a technical standpoint, it’s nice to recognize the distinction. You may see require used in its place. If the package is not installed, library will trigger an error while require will return FALSE. Once loaded the functions created in the package are available to your R session.

library(Hmisc)
require(Hmisc)

Fetching Data, Modifying Variables, and Printing Data Dictionary

The getHdata function is used to fetch a dataset from the Vanderbilt DataSets web site. upData is used to

contents is used to print a data dictionary, run through an html method for nicer output.

getHdata(pbc)
pbc <- upData(pbc,
              fu.yrs = fu.days / 365.25,
              labels = c(fu.yrs = 'Follow-up Time',
                         status = 'Death or Liver Transplantation'),
              units = c(fu.yrs = 'year'),
              drop  = 'fu.days',
              moveUnits=TRUE, html=TRUE)
Input object size:   76592 bytes;    19 variables    418 observations
 Label for bili changed from Serum Bilirubin (mg/dl) to Serum Bilirubin 
    units set to mg/dl 
 Label for albumin changed from Albumin (gm/dl) to Albumin 
    units set to gm/dl 
 Label for protime changed from Prothrombin Time (sec.) to Prothrombin Time 
    units set to sec. 
 Label for alk.phos changed from Alkaline Phosphatase (U/liter) to Alkaline Phosphatase 
    units set to U/liter 
 Label for sgot changed from SGOT (U/ml) to SGOT 
    units set to U/ml 
 Label for chol changed from Cholesterol (mg/dl) to Cholesterol 
    units set to mg/dl 
 Label for trig changed from Triglycerides (mg/dl) to Triglycerides 
    units set to mg/dl 
 Label for platelet changed from Platelets (per cm^3/1000) to Platelets 
    units set to per cm^3/1000 
 Label for copper changed from Urine Copper (ug/day) to Urine Copper 
    units set to ug/day 
 Added variable     fu.yrs
 Dropped variable   fu.days
 New object size:   80224 bytes;    19 variables    418 observations
html(contents(pbc), maxlevels=10, levelType='table')

Data frame:pbc

418 observations and 19 variables, maximum # NAs:136  
NameLabelsUnitsLevelsStorageNAs
biliSerum Bilirubinmg/dldouble 0
albuminAlbumingm/dldouble 0
stageHistologic Stage, Ludwig Criteriainteger 6
protimeProthrombin Timesec.double 2
sex2integer 0
ageAgedouble 0
spiders2integer106
hepatom2integer106
ascites2integer106
alk.phosAlkaline PhosphataseU/literdouble106
sgotSGOTU/mldouble106
cholCholesterolmg/dlinteger134
trigTriglyceridesmg/dlinteger136
plateletPlateletsper cm^3/1000integer110
drug3integer 0
statusDeath or Liver Transplantationinteger 0
edema3integer 0
copperUrine Copperug/dayinteger108
fu.yrsFollow-up Timeyeardouble 0

VariableLevels
sexmale
female
spiders, hepatomabsent
 ascitespresent
drugD-penicillamine
placebo
not randomized
edemano edema
edema, no diuretic therapy
edema despite diuretic therapy

Descriptive Statistics Without Stratification

# did have results='asis' above
d <- describe(pbc)
html(d, size=80, scroll=TRUE)

pbc

19 Variables   418 Observations

bili: Serum Bilirubin mg/dl
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
4180980.9983.2213.742 0.50 0.60 0.80 1.40 3.40 8.0314.00
lowest : 0.3 0.4 0.5 0.6 0.7 , highest: 21.6 22.5 24.5 25.5 28.0
albumin: Albumin gm/dl
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
418015413.4970.4732.7502.9673.2433.5303.7704.0104.141
lowest : 1.96 2.10 2.23 2.27 2.31 , highest: 4.30 4.38 4.40 4.52 4.64
stage: Histologic Stage, Ludwig Criteria
image
nmissingdistinctInfoMeanGmd
412640.8933.0240.9519
 Value          1     2     3     4
 Frequency     21    92   155   144
 Proportion 0.051 0.223 0.376 0.350
 

protime: Prothrombin Time sec.
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
4162480.99810.731.029 9.60 9.8010.0010.6011.1012.0012.45
lowest : 9.0 9.1 9.2 9.3 9.4 , highest: 13.8 14.1 15.2 17.1 18.0
sex
nmissingdistinct
41802
 Value        male female
 Frequency      44    374
 Proportion  0.105  0.895
 

age: Age
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
4180345150.7411.9633.8436.3742.8351.0058.2464.3067.92
lowest : 26.27789 28.88433 29.55510 30.27515 30.57358 , highest: 74.52430 75.00000 75.01164 76.70910 78.43943
spiders
nmissingdistinct
3121062
 Value       absent present
 Frequency      222      90
 Proportion   0.712   0.288
 

hepatom
nmissingdistinct
3121062
 Value       absent present
 Frequency      152     160
 Proportion   0.487   0.513
 

ascites
nmissingdistinct
3121062
 Value       absent present
 Frequency      288      24
 Proportion   0.923   0.077
 

alk.phos: Alkaline Phosphatase U/liter
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
312106295119831760 599.6 663.0 871.51259.01980.03826.46669.9
lowest : 289.0 310.0 369.0 377.0 414.0 , highest: 11046.6 11320.2 11552.0 12258.8 13862.4
sgot: SGOT U/ml
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
3121061791122.660.45 54.25 60.45 80.60114.70151.90196.47219.25
lowest : 26.35 28.38 41.85 43.40 45.00 , highest: 288.00 299.15 328.60 338.00 457.25
chol: Cholesterol mg/dl
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
2841342011369.5194.5188.4213.6249.5309.5400.0560.8674.0
lowest : 120 127 132 149 151 , highest: 1336 1480 1600 1712 1775
trig: Triglycerides mg/dl
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
2821361461124.764.07 56.00 63.10 84.25108.00151.00195.00230.95
lowest : 33 44 46 49 50 , highest: 319 322 382 432 598
platelet: Platelets per cm3/1000
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
3081102101261.9107.8117.7139.7199.8257.0322.5386.5430.6
lowest : 62 70 71 79 80 , highest: 493 514 518 539 563
drug
image
nmissingdistinct
41803
 Value      D-penicillamine         placebo  not randomized
 Frequency              154             158             106
 Proportion           0.368           0.378           0.254
 

status: Death or Liver Transplantation
nmissingdistinctInfoSumMeanGmd
418020.711610.38520.4748

edema
image
nmissingdistinct
41803
 Value                            no edema     edema, no diuretic therapy
 Frequency                             354                             44
 Proportion                          0.847                          0.105
                                          
 Value      edema despite diuretic therapy
 Frequency                              20
 Proportion                          0.048
 

copper: Urine Copper ug/day
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
310108158197.6583.16 17.45 24.00 41.25 73.00123.00208.10249.20
lowest : 4 9 10 11 12 , highest: 412 444 464 558 588
fu.yrs: Follow-up Time year
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
418039915.2513.429 0.671 1.661 2.992 4.736 7.155 9.64911.063
lowest : 0.1122519 0.1177276 0.1396304 0.1943874 0.2108145
highest:12.320328512.344969212.383299112.473648213.1279945

plot(d)

$Categorical

$Continuous
NA

Stratified Descriptive Statistics

Produce stratified quantiles, means/SD, and proportions by treatment group. Plot the results before rendering as an advanced html table:

s <- summaryM(bili + albumin + stage + protime + sex + age + spiders +
              alk.phos + sgot + chol ~ drug, data=pbc,
                            overall=FALSE, test=TRUE)
plot(s, which='categorical')

Ignoring 1 observationsIgnoring 1 observations

plot(s, which='continuous', vars=1:4)

Ignoring 3 observationsIgnoring 3 observationsIgnoring 3 observationsIgnoring 3 observations

plot(s, which='continuous', vars=5:7)

Ignoring 2 observationsIgnoring 2 observationsIgnoring 2 observations

html(s, caption='Baseline characteristics by randomized treatment',
     exclude1=TRUE, npct='both', digits=3,
     prmsd=TRUE, brmsd=TRUE, msdsize=mu$smaller2)
Baseline characteristics by randomized treatment.
N
D-penicillamine
N=154
placebo
N=158
not randomized
N=106
Test Statistic
Serum Bilirubin
mg/dl
418 0.725 1.300 3.600
3.649 ± 5.282
0.800 1.400 3.200
2.873 ± 3.629
0.725 1.400 3.075
3.117 ± 4.043
F2 415=0.03, P=0.9721
Albumin
gm/dl
418 3.342 3.545 3.777
3.524 ± 0.396
3.212 3.565 3.830
3.516 ± 0.443
3.125 3.470 3.720
3.431 ± 0.435
F2 415=2.13, P=0.121
Histologic Stage, Ludwig Criteria : 1 412 0.03 4154 0.08 12158 0.05 5100 χ26=5.33, P=0.5022
  2 0.21 32154 0.22 35158 0.25 25100
  3 0.42 64154 0.35 56158 0.35 35100
  4 0.35 54154 0.35 55158 0.35 35100
Prothrombin Time
sec.
416 10.000 10.600 11.400
10.800 ±  1.138
10.025 10.600 11.000
10.653 ±  0.851
10.100 10.600 11.000
10.750 ±  1.078
F2 413=0.23, P=0.7951
sex : female 418 0.90 139154 0.87 137158 0.92 98106 χ22=2.38, P=0.3042
Age 418 41.43 48.11 55.80
48.58 ±  9.96
42.98 51.93 58.90
51.42 ± 11.01
46.00 53.00 61.00
52.87 ±  9.78
F2 415=6.1, P=0.0021
spiders 312 0.29 45154 0.28 45158 χ21=0.02, P=0.8852
Alkaline Phosphatase
U/liter
312 922 1283 1950
1943 ± 2102
841 1214 2028
2021 ± 2183
F1 310=0.06, P=0.8123
SGOT
U/ml
312 83.8 117.4 151.9
125.0 ±  58.9
76.7 111.6 151.5
120.2 ±  54.5
F1 310=0.55, P=0.463
Cholesterol
mg/dl
284 254 304 377
374 ± 252
248 316 417
365 ± 210
F1 282=0.37, P=0.5453
a b c represent the lower quartile a, the median b, and the upper quartile c for continuous variables. x ± s represents X ± 1 SD.   N is the number of non-missing values.
Tests used: 1Kruskal-Wallis test; 2Pearson test; 3Wilcoxon test .

Spike Histogram

p <- with(pbc, histboxp(x=sgot, group=drug, sd=TRUE))
p

Data Visualization

The very low birthweight data set contains data on 671 infants born with a birth weight of under 1600 grams. We’ll plot gestational age by birthweight using three graphics systems: base graphics, ggplot, and plotly.

getHdata(vlbw)
# remove missing values
vlbw <- vlbw[complete.cases(vlbw[,c('sex','dead','gest','bwt')]),]

Base

Build each element into your plot.

grps <- split(vlbw[,c('gest','bwt')], vlbw[,c('sex','dead')])
plot(c(22,40), c(400,1600), type='n', xlab='Gestational Age', ylab='Birth Weight (grams)', axes=FALSE)
axis(1, at=c(22,28,34,40), labels=c(22,28,34,40))
axis(2, at=seq(400,1600,by=400), labels=seq(400,1600,by=400))
points(grps[['female.0']], col='black', pch=1)
points(grps[['female.1']][,'gest'], grps[['female.1']][,'bwt'], col='black', pch=0)
points(jitter(grps[['male.0']][,'gest'], 2), grps[['male.0']][,'bwt'], col='gray', pch=4)
points(grps[['male.0']][,'bwt'] ~ jitter(grps[['male.0']][,'gest'], 2), col='gray', pch=3)
legend(x=38, y=1000, legend=c('F:0','F:1','M:0','M:1'), col=c('black','black','gray','gray'), pch=c(1,0,4,3))

ggplot2

Given a data set, choose the aesthetic mapping and geometry layer.

p <- ggplot(data=vlbw) + aes(x=gest, y=bwt, color=sex, shape=as.factor(dead)) + geom_point()
p

p <- p + geom_jitter() + scale_x_continuous() + scale_y_continuous()
p

Plotly

Add interactive graphics, which is trivial when also using ggplot.

require(plotly)
Loading required package: plotly

Attaching package: ‘plotly’

The following object is masked from ‘package:Hmisc’:

    subplot

The following object is masked from ‘package:ggplot2’:

    last_plot

The following object is masked from ‘package:stats’:

    filter

The following object is masked from ‘package:graphics’:

    layout
ggplotly(p)
plot_ly(type="box") %>%
  add_boxplot(y = grps[['female.0']][,'bwt'], name='F:0') %>%
  add_boxplot(y = grps[['female.1']][,'bwt'], name='F:1') %>%
  add_boxplot(y = grps[['male.0']][,'bwt'], name='M:0') %>%
  add_boxplot(y = grps[['male.1']][,'bwt'], name='M:1')

Cardiovascular risk factor data

getHdata(diabetes)
html(contents(diabetes), levelType='table')

Data frame:diabetes

403 observations and 19 variables, maximum # NAs:262  
NameLabelsUnitsLevelsStorageNAs
idSubject IDinteger 0
cholTotal Cholesterolinteger 1
stab.gluStabilized Glucoseinteger 0
hdlHigh Density Lipoproteininteger 1
ratioCholesterol/HDL Ratiodouble 1
glyhbGlycosolated Hemoglobindouble 13
location2integer 0
ageyearsinteger 0
gender2integer 0
heightinchesinteger 5
weightpoundsinteger 1
frame3integer 12
bp.1sFirst Systolic Blood Pressureinteger 5
bp.1dFirst Diastolic Blood Pressureinteger 5
bp.2sSecond Systolic Blood Pressureinteger262
bp.2dSecond Diastolic Blood Pressureinteger262
waistinchesinteger 2
hipinchesinteger 2
time.ppnPostprandial Time when Labs were Drawnminutesinteger 3

VariableLevels
locationBuckingham
Louisa
gendermale
female
framesmall
medium
large

Formulas

The tilde is used to create a model formula, which consists of a left-hand side and right-hand side. Many R functions utilize formulas, such as plotting functions and model-fitting functions. The left-hand side consists of the response variable, while the right-hand side may contain several terms. You may see the following operators within a formula.

See ?formula for more examples.

Models

The most simple model-fitting function is lm, which is used to fit linear models. It’s primary argument is a formula. Using the diabetes data set, we can fit waist size by weight.

(m <- lm(waist ~ weight, data=diabetes))

Call:
lm(formula = waist ~ weight, data = diabetes)

Coefficients:
(Intercept)       weight  
    16.5050       0.1205  

This creates an lm object, and several functions can be used on model objects. The internal structure of a model object is a list - its elements may be accessed just like a list.

names(m)
 [1] "coefficients"  "residuals"     "effects"       "rank"          "fitted.values" "assign"        "qr"           
 [8] "df.residual"   "na.action"     "xlevels"       "call"          "terms"         "model"        
m$coefficients
(Intercept)      weight 
 16.5050329   0.1204712 
coef(m)
(Intercept)      weight 
 16.5050329   0.1204712 
head(fitted(m))
       1        2        3        4        5        6 
31.08205 42.76776 47.34567 30.84111 38.55127 39.39457 
predict(m, data.frame(weight=c(150, 200, 250)))
       1        2        3 
34.57572 40.59928 46.62284 
head(residuals(m))
        1         2         3         4         5         6 
-2.082053  3.232237  1.654330  2.158890  5.448731 -3.394568 
vcov(m)
             (Intercept)        weight
(Intercept)  0.465908502 -0.0024927458
weight      -0.002492746  0.0000140231
summary(m)

Call:
lm(formula = waist ~ weight, data = diabetes)

Residuals:
    Min      1Q  Median      3Q     Max 
-7.6835 -2.0850 -0.2021  1.7741 11.5330 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept) 16.505033   0.682575   24.18   <2e-16 ***
weight       0.120471   0.003745   32.17   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 3.02 on 398 degrees of freedom
  (3 observations deleted due to missingness)
Multiple R-squared:  0.7223,    Adjusted R-squared:  0.7216 
F-statistic:  1035 on 1 and 398 DF,  p-value: < 2.2e-16
coef(summary(m))
              Estimate  Std. Error  t value      Pr(>|t|)
(Intercept) 16.5050329 0.682574906 24.18054  3.970142e-80
weight       0.1204712 0.003744743 32.17077 9.082965e-113
anova(m)
Analysis of Variance Table

Response: waist
           Df Sum Sq Mean Sq F value    Pr(>F)    
weight      1 9438.0  9438.0    1035 < 2.2e-16 ***
Residuals 398 3629.4     9.1                      
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Visualization

Create a scatterplot of weight and waist size.

p <- qplot(weight, waist, data=diabetes)
p + geom_smooth(method="lm")

Remove missing values and fit with LOESS curve.

diab <- diabetes[complete.cases(diabetes[,c('waist','weight')]),]
p <- qplot(weight, waist, data=diab)
p + geom_smooth(method="loess")

Data Manipulation

The Dominican Republic Hypertension dataset contains data on gender, age, and systolic and diastolic blood pressure from several villages in the DR.

getHdata(DominicanHTN)
html(describe(DominicanHTN))
DominicanHTN

5 Variables   381 Observations

village
image
nmissingdistinct
38108
 Value        Bare' Nuevo   Batey Verde       Carmona       Cojobal  Juan Sanchez
 Frequency             32            41            64            59            53
 Proportion         0.084         0.108         0.168         0.155         0.139
                                                     
 Value      La Altagracia   Los Gueneos   San Antonio
 Frequency             40            57            35
 Proportion         0.105         0.150         0.092
 

gender
nmissingdistinct
38102
 Value      Female   Male
 Frequency     258    123
 Proportion  0.677  0.323
 

age
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
3810690.99947.9716.9823303847596873
lowest : 15 17 18 19 20 , highest: 85 87 90 95 100
sbp: Systolic Blood Pressure mmHg
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
3810570.99613328.35 98105118130150170180
lowest : 80 90 95 98 99 , highest: 196 200 204 210 236
dbp: Diastolic Blood Pressure mmHg
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
3810410.99384.2315.58 64 70 76 82 92100110
lowest : 20 40 50 60 62 , highest: 115 118 120 130 152

Adding variables or transformations

DominicanHTN[,'map'] <- (DominicanHTN[,'sbp'] + DominicanHTN[,'dbp'] * 2) / 3
DominicanHTN[,'male'] <- as.numeric(DominicanHTN[,'gender'] == 'Male')
DominicanHTN[1:5,]
qage <- quantile(DominicanHTN[,'age'])
qage[1] <- qage[1]-1
DominicanHTN[,'ageGrp'] <- cut(DominicanHTN[,'age'], breaks=qage)

Filter

nrow(DominicanHTN)
[1] 381
nrow(DominicanHTN[DominicanHTN[,'gender'] == "Male",])
[1] 123
which(DominicanHTN[,'village'] %in% c('Carmona','San Antonio'))
 [1] 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143
[32] 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174
[63] 175 176 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277
[94] 278 279 280 281 282 283
cojMen <- DominicanHTN[DominicanHTN[,'village'] == 'Cojobal' & DominicanHTN[,'gender'] == "Male",]
cojMen

Sort

cojMen[order(cojMen[,'age'], decreasing=TRUE),]

Aggregate and counts

table(DominicanHTN[,'gender'])

Female   Male 
   258    123 
addmargins(with(DominicanHTN, table(village, gender)))
               gender
village         Female Male Sum
  Bare' Nuevo       19   13  32
  Batey Verde       30   11  41
  Carmona           39   25  64
  Cojobal           47   12  59
  Juan Sanchez      40   13  53
  La Altagracia     22   18  40
  Los Gueneos       41   16  57
  San Antonio       20   15  35
  Sum              258  123 381
with(DominicanHTN, tapply(age, village, mean))
  Bare' Nuevo   Batey Verde       Carmona       Cojobal  Juan Sanchez La Altagracia   Los Gueneos   San Antonio 
     47.53125      48.02439      49.00000      47.15254      47.45283      49.65000      45.15789      51.25714 
aggregate(sbp ~ gender, data=DominicanHTN, FUN=mean)
aggregate(age ~ village + gender, DominicanHTN, median)

Rosner’s lead data

The lead exposure dataset was collected to study the well-being of childen who lived near a lead smelting plant. The following analysis considers the lead exposure levels in 1972 and 1973, as well as age and the finger-wrist tapping score maxfwt.

getHdata(lead)
lead <- upData(lead,
       keep = c('ld72','ld73','age','maxfwt'),
       labels = c(age = 'Age'),
       units = c(age='years', ld72='mg/100*ml', ld73='mg/100*ml'))

Input object size: 50832 bytes; 39 variables 124 observations Kept variables ld72,ld73,age,maxfwt New object size: 12824 bytes; 4 variables 124 observations

html(contents(lead), maxlevels=10, levelType='table')

Data frame:lead

124 observations and 4 variables, maximum # NAs:25  
NameLabelsUnitsStorageNAs
ageAgeyearsdouble 0
ld72Blood Lead Levels, 1972mg/100*mlinteger 0
ld73Blood Lead Levels, 1973mg/100*mlinteger 0
maxfwtMaximum mean finger-wrist tapping scoreinteger25

html(describe(lead))
lead

4 Variables   124 Observations

age: Age years
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
12407318.9354.074 3.929 4.333 6.167 8.37512.02114.00015.000
lowest : 3.750000 3.833333 3.916667 4.000000 4.166667
highest:14.25000015.00000015.25000015.41666715.916667

ld72: Blood Lead Levels, 1972 mg/100*ml
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
1240470.99936.1617.2318.0021.0027.0034.0043.0057.0061.85
lowest : 1 2 10 14 18 , highest: 62 64 66 68 99
ld73: Blood Lead Levels, 1973 mg/100*ml
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
1240370.99831.7111.0618.1521.0024.0030.5037.0047.0050.85
lowest : 15 16 18 19 20 , highest: 52 53 54 57 58
maxfwt: Maximum mean finger-wrist tapping score
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
9925400.99851.9613.833.238.046.052.059.065.072.2
lowest : 13 14 23 26 34 , highest: 74 76 79 83 84

Ordinary least squares regression

The rms package includes the ols fitting function. Use datadist to compute summaries of the distributional charateristics of the predictors - or simply give it an entire data frame. datadist must be re-run if you add a new predictor or recode an old one.

require(rms)
Loading required package: rms
Loading required package: SparseM

Attaching package: ‘SparseM’

The following object is masked from ‘package:base’:

    backsolve
dd <- datadist(lead)
options(datadist='dd')
f <- ols(maxfwt ~ age + ld72 + ld73, data=lead)
Calling 'structure(NULL, *)' is deprecated, as NULL cannot have attributes.
  Consider 'structure(list(), *)' instead.
f
Frequencies of Missing Values Due to Each Variable
maxfwt    age   ld72   ld73 
    25      0      0      0 

Linear Regression Model
 
 ols(formula = maxfwt ~ age + ld72 + ld73, data = lead)
 
 
                Model Likelihood     Discrimination    
                   Ratio Test           Indexes        
 Obs      99    LR chi2     62.25    R2       0.467    
 sigma9.5221    d.f.            3    R2 adj   0.450    
 d.f.     95    Pr(> chi2) 0.0000    g       10.104    
 
 Residuals
 
      Min       1Q   Median       3Q      Max 
 -33.9958  -4.9214   0.7596   5.1106  33.2590 
 
 
           Coef    S.E.   t     Pr(>|t|)
 Intercept 34.1059 4.8438  7.04 <0.0001 
 age        2.6078 0.3231  8.07 <0.0001 
 ld72      -0.0246 0.0782 -0.31 0.7538  
 ld73      -0.2390 0.1325 -1.80 0.0744  
 

rms provides many methods to work with the ols fit object.

Coefficient Estimates

coef(f)
 Intercept        age       ld72       ld73 
34.1058551  2.6078450 -0.0245978 -0.2389695 

Define R function representing fitted model

The default values for function arguments are medians.

g <- Function(f)
g
function(age = 8.375,ld72 = 34,ld73 = 30.5) {34.105855+2.607845*age-0.024597799*ld72-0.23896951*ld73 }
<environment: 0x1002f268>
g(age=10, ld72=21, ld73=c(21, 47))
[1] 54.64939 48.43618

Predicted values with standard errors

predict(f, data.frame(age=10, ld72=21, ld73=c(21, 47)), se.fit=TRUE)
$linear.predictors
       1        2 
54.64939 48.43618 

$se.fit
       1        2 
1.391858 3.140361 

Residuals

r <- resid(f)
par(mfrow=c(2,2))
plot(fitted(f), r); abline(h=0)
with(lead, plot(age, r)); abline(h=0)
with(lead, plot(ld73, r)); abline(h=0)
# q-q plot to check normality
qqnorm(r)

Plotting partial effects

Predict and pplot make one plot for each predictor. Predictors not shown in plot are set to constants (continuous:median, categorical:mode). 0.95 pointwise confidence limits for \(\hat{E}(y|x)\) are shown; use conf.int=FALSE to suppress CLs.

plotp(Predict(f))

Specify which predictors are plotted.

plotp(Predict(f, age))
plotp(Predict(f, age=3:15))
plotp(Predict(f, age=seq(3,16,length=150)))

Obtain confidence limits for \(\hat{y}\).

plotp(Predict(f, age, conf.type='individual'))

Combine plots.

p1 <- Predict(f, age, conf.int=0.99, conf.type='individual')
p2 <- Predict(f, age, conf.int=0.99, conf.type='mean')
p <- rbind(Individual=p1, Mean=p2)
ggplot(p)
plotp(Predict(f, ld73, age=3))
ggplot(Predict(f, ld73, age=c(3,9)))

3-d surface for two continuous predictors against \(\hat{y}\).

bplot(Predict(f, ld72, ld73))

Nomograms

plot(nomogram(f))

Point Estimates for Partial Effects

summary(f)
             Effects              Response : maxfwt 

 Factor Low     High   Diff.   Effect   S.E.   Lower 0.95 Upper 0.95
 age     6.1667 12.021  5.8542 15.26700 1.8914 11.5120    19.02200  
 ld72   27.0000 43.000 16.0000 -0.39356 1.2511 -2.8773     2.09010  
 ld73   24.0000 37.000 13.0000 -3.10660 1.7222 -6.5255     0.31234  

Adjust age to 5, which has no effect as the model does not include any interactions.

summary(f, age=5)
             Effects              Response : maxfwt 

 Factor Low     High   Diff.   Effect   S.E.   Lower 0.95 Upper 0.95
 age     6.1667 12.021  5.8542 15.26700 1.8914 11.5120    19.02200  
 ld72   27.0000 43.000 16.0000 -0.39356 1.2511 -2.8773     2.09010  
 ld73   24.0000 37.000 13.0000 -3.10660 1.7222 -6.5255     0.31234  

Effect of changing ld73 from 20 to 40.

summary(f, ld73=c(20,40))
             Effects              Response : maxfwt 

 Factor Low     High   Diff.   Effect   S.E.   Lower 0.95 Upper 0.95
 age     6.1667 12.021  5.8542 15.26700 1.8914  11.5120   19.02200  
 ld72   27.0000 43.000 16.0000 -0.39356 1.2511  -2.8773    2.09010  
 ld73   20.0000 40.000 20.0000 -4.77940 2.6495 -10.0390    0.48052  

If a predictor has a linear effect, a one-unit change can be used to get the confidence interval of its slope.

summary(f, age=5:6)
             Effects              Response : maxfwt 

 Factor Low High Diff. Effect   S.E.    Lower 0.95 Upper 0.95
 age     5   6    1     2.60780 0.32308  1.9664    3.24920   
 ld72   27  43   16    -0.39356 1.25110 -2.8773    2.09010   
 ld73   24  37   13    -3.10660 1.72220 -6.5255    0.31234   

There is also a plot method for summary results.

plot(summary(f))

Getting Predicted Values

Give predict a fit object and data.frame.

predict(f, data.frame(age=3, ld72=21, ld73=21))
       1 
36.39448 
predict(f, data.frame(age=c(3,10), ld72=21, ld73=c(21,47)))
       1        2 
36.39448 48.43618 
newdat <- expand.grid(age=c(4,8), ld72=c(21, 47), ld73=c(21, 47))
newdat
predict(f, newdat)
       1        2        3        4        5        6        7        8 
39.00232 49.43370 38.36278 48.79416 32.78911 43.22049 32.14957 42.58095 

Include confidence levels.

predict(f, newdat, conf.int=0.95)
$linear.predictors
       1        2        3        4        5        6        7        8 
39.00232 49.43370 38.36278 48.79416 32.78911 43.22049 32.14957 42.58095 

$lower
       1        2        3        4        5        6        7        8 
33.97441 46.23595 32.15468 43.94736 25.68920 36.94167 27.17060 38.86475 

$upper
       1        2        3        4        5        6        7        8 
44.03023 52.63145 44.57088 53.64095 39.88902 49.49932 37.12854 46.29716 
predict(f, newdat, conf.int=0.95, conf.type='individual')
$linear.predictors
       1        2        3        4        5        6        7        8 
39.00232 49.43370 38.36278 48.79416 32.78911 43.22049 32.14957 42.58095 

$lower
       1        2        3        4        5        6        7        8 
19.44127 30.26132 18.46566 29.27888 12.59596 23.30120 12.60105 23.31531 

$upper
       1        2        3        4        5        6        7        8 
58.56337 68.60609 58.25989 68.30944 52.98227 63.13979 51.69810 61.84659 

Brute-force predicted values.

b <- coef(f)
b[1] + b[2]*3 + b[3]*21 + b[4]*21
Intercept 
 36.39448 

Predicted values with Function.

# g <- Function(f)
g(age = c(3,8), ld72 = 21, ld73 = 21)
[1] 36.39448 49.43370
g(age = 3)
[1] 33.80449

ANOVA

Use anova to get all total effects and individual partial effects.

an <- anova(f)
an
                Analysis of Variance          Response: maxfwt 

 Factor     d.f. Partial SS  MS          F     P     
 age         1   5907.535742 5907.535742 65.15 <.0001
 ld72        1      8.972994    8.972994  0.10 0.7538
 ld73        1    295.044370  295.044370  3.25 0.0744
 REGRESSION  3   7540.087710 2513.362570 27.72 <.0001
 ERROR      95   8613.750674   90.671060             

Include corresponding variable names.

print(an, 'names')
                Analysis of Variance          Response: maxfwt 

 Factor     d.f. Partial SS  MS          F     P      Tested       
 age         1   5907.535742 5907.535742 65.15 <.0001 age          
 ld72        1      8.972994    8.972994  0.10 0.7538 ld72         
 ld73        1    295.044370  295.044370  3.25 0.0744 ld73         
 REGRESSION  3   7540.087710 2513.362570 27.72 <.0001 age,ld72,ld73
 ERROR      95   8613.750674   90.671060                           
print(an, 'subscripts')
                Analysis of Variance          Response: maxfwt 

 Factor     d.f. Partial SS  MS          F     P      Tested
 age         1   5907.535742 5907.535742 65.15 <.0001 1     
 ld72        1      8.972994    8.972994  0.10 0.7538 2     
 ld73        1    295.044370  295.044370  3.25 0.0744 3     
 REGRESSION  3   7540.087710 2513.362570 27.72 <.0001 1-3   
 ERROR      95   8613.750674   90.671060                    

Subscripts correspond to:
[1] age  ld72 ld73
print(an, 'dots')
                Analysis of Variance          Response: maxfwt 

 Factor     d.f. Partial SS  MS          F     P      Tested
 age         1   5907.535742 5907.535742 65.15 <.0001 .     
 ld72        1      8.972994    8.972994  0.10 0.7538  .    
 ld73        1    295.044370  295.044370  3.25 0.0744   .   
 REGRESSION  3   7540.087710 2513.362570 27.72 <.0001 ...   
 ERROR      95   8613.750674   90.671060                    

Subscripts correspond to:
[1] age  ld72 ld73

Combined partial effects.

anova(f, ld72, ld73)
                Analysis of Variance          Response: maxfwt 

 Factor     d.f. Partial SS  MS         F    P     
 ld72        1      8.972994   8.972994 0.10 0.7538
 ld73        1    295.044370 295.044370 3.25 0.0744
 REGRESSION  2    747.283558 373.641779 4.12 0.0192
 ERROR      95   8613.750674  90.671060            

Importing Data

Text files

  • read.table - work with data in table format
  • read.csv - CSV files
  • read.delim - tab-delimited files
  • scan - more general function for reading in data

Binary representation of R objects

Compressed data and loads faster.

  • RData
  • feather package

Database connections

Allows data queries.

  • RODBC package
  • RSQLite package

Other statistical packages

  • Hmisc package: sas.get, sasxport.get
  • haven package: read_sas, read_sav, read_dta
  • foreign package
  • Stat/Transfer - not free nor open source

Data Cleaning

An R data.frame consists of a collection of values. In a well-structured data set, each value is associated with a variable (column) and observation (row). Data sets often need to be manipulated to become well-structured.

Hadley Wickham’s Tidy Data provides an excellent overview on how to clean messy data sets.

Column header contains values

# politics and religion
senateReligion <- data.frame(religion=c('Protestant','Catholic','Jewish','Mormon','Buddhist',"Don't Know/Refused"),
                          Democrats=c(20,15,8,1,1,3),
                          Republicans=c(38,9,0,5,0,0))
senRel <- cbind(senateReligion[,'religion'], stack(senateReligion[,c('Democrats','Republicans')]))
names(senRel) <- c('religion','count','party')
senRel

Multiple variables in a column

pets <- data.frame(county=c('Davidson','Rutherford','Cannon'),
                   male.dog=c(50,150,70), female.dog=c(60,150,70),
                   male.cat=c(30,100,50), female.cat=c(30,70,40),
                   male.horse=c(6,30,20), female.horse=c(6,28,19))
pets2 <- cbind(pets[,'county'], stack(pets[,-1]))
names(pets2) <- c('county','count','ind')

Watch out for factor variables.

tryCatch(strsplit(pets2[,'ind'], '\\.'), error=function(e){ e })
<simpleError in strsplit(pets2[, "ind"], "\\."): non-character argument>
genderAnimal <- strsplit(as.character(pets2[,'ind']), '\\.')
pets2[,c('gender','animal')] <- NA
for(i in seq(nrow(pets2))) {
  pets2[i, c('gender','animal')] <- genderAnimal[[i]]
}
pets2[,'ind'] <- NULL
pets2

Variables in a row

set.seed(1000)
d1 <- rnorm(1000, 5, 2)
d2 <- runif(1000, 0, 10)
d3 <- rpois(1000, 5)
d4 <- rbinom(1000, 10, 0.5)
x <- data.frame(distribution=c(rep('normal',2),rep('uniform',2),rep('poisson',2),rep('binomial',2)),
           stat=rep(c('mean','sd'),4),
           value=c(mean(d1), sd(d1), mean(d2), sd(d2), mean(d3), sd(d3), mean(d4), sd(d4)))
cbind(distribution=x[c(1,3,5,7),'distribution'], unstack(x, form=value~stat))

Normalization and merging

employees <- data.frame(id=1:4, Name=c('Eddie','Andrea','Steve','Theresa'),
                        job=c('engineer','accountant','statistician','technician'))
set.seed(10)
hours <- data.frame(id=sample(4, 10, replace=TRUE),
                    week=1:10, hours=sample(30:60, 10, replace=TRUE))
merge(employees, hours)
# merge(employees, hours, by.x='id', by.y='id')
empHours <- merge(employees, hours, all.x=TRUE)
empHours
unique(empHours[,c('id','Name','job')])
empHours[,c('id','week','hours')]

Exclude rows with missing data.

empHours[!is.na(empHours[,'week']), c('id','week','hours')]

Combining data

rbind(genderAnimal[[1]], genderAnimal[[2]], genderAnimal[[3]])
     [,1]   [,2] 
[1,] "male" "dog"
[2,] "male" "dog"
[3,] "male" "dog"
do.call(rbind, genderAnimal)
      [,1]     [,2]   
 [1,] "male"   "dog"  
 [2,] "male"   "dog"  
 [3,] "male"   "dog"  
 [4,] "female" "dog"  
 [5,] "female" "dog"  
 [6,] "female" "dog"  
 [7,] "male"   "cat"  
 [8,] "male"   "cat"  
 [9,] "male"   "cat"  
[10,] "female" "cat"  
[11,] "female" "cat"  
[12,] "female" "cat"  
[13,] "male"   "horse"
[14,] "male"   "horse"
[15,] "male"   "horse"
[16,] "female" "horse"
[17,] "female" "horse"
[18,] "female" "horse"

Simulation

Plot mean while sampling from normal distribution

n <- 100
imean <- numeric(n)
smean <- numeric(n)
for(i in seq(n)) {
  imean[i] <- mean(rnorm(10))
  smean[i] <- mean(imean[seq(i)])
}
plot(imean, ylab='Sample Mean')
abline(h=0, lty=3)
lines(smean)

n <- 1000
sig <- numeric(n)
for(i in seq(n)) {
  grp1 <- rnorm(15, 60, 5)
  grp2 <- rnorm(15, 65, 5)
  sig[i] <- t.test(grp1, grp2, var.equal = TRUE)$p.value < 0.05
}
mean(sig)
[1] 0.723

Add some flexibility to our simulation by creating a function.

tTestPower <- function(n=15, mu1=60, mu2=65, sd=5, nsim=1000) {
  sig <- numeric(nsim)
  for(i in seq(nsim)) {
    grp1 <- rnorm(n, mu1, sd)
    grp2 <- rnorm(n, mu2, sd)
    sig[i] <- t.test(grp1, grp2, var.equal = TRUE)$p.value < 0.05
  }
  mean(sig)
}
tTestPower()
[1] 0.758
tTestPower(25, nsim=10000)
[1] 0.9306

There’s already a function to compute the power of a two-sample t test.

power.t.test(n=25, delta=5, sd=5)$power
[1] 0.9337076

Bootstrap sample

true_mu <- 0
x <- rnorm(100, true_mu)
R <- 999
res <- matrix(nrow=R, ncol=6)
colnames(res) <- c('mu','se','lb','ub','coverage','bias')
for(i in seq(R)) {
  r <- sample(x, replace=TRUE)
  res[i,'mu'] <- mean(r)
  res[i,'se'] <- sd(r) / sqrt(length(r))
}
res[,'lb'] <- res[,'mu'] + qnorm(0.025) * res[,'se']
res[,'ub'] <- res[,'mu'] + qnorm(0.975) * res[,'se']
res[,'coverage'] <- res[,'lb'] < true_mu & res[,'ub'] > true_mu
res[,'bias'] <- res[,'mu'] - true_mu
res[1:5,]
               mu         se          lb        ub coverage         bias
[1,]  0.004925515 0.10318672 -0.19731674 0.2071678        1  0.004925515
[2,]  0.102092092 0.08152406 -0.05769213 0.2618763        1  0.102092092
[3,] -0.015542538 0.08834944 -0.18870427 0.1576192        1 -0.015542538
[4,] -0.040089989 0.10071664 -0.23749097 0.1573110        1 -0.040089989
[5,] -0.047656957 0.10017721 -0.24400068 0.1486868        1 -0.047656957
vals <- colMeans(res)
basicCI <- 2 * mean(x) - quantile(res[,'mu'], probs=c(0.975, 0.025))
out <- sprintf("empirical SE: %.3f
MSE: %.3f
basic bootstrap confidence interval: (%.3f, %.3f)
coverage probability: %.3f
mean bias: %.3f
sample mean: %.3f
", sd(res[,'mu']), vals['se'], basicCI[1], basicCI[2], vals['coverage'], vals['bias'], mean(x))
cat(out)
empirical SE: 0.100
MSE: 0.099
basic bootstrap confidence interval: (-0.212, 0.188)
coverage probability: 0.933
mean bias: -0.023
sample mean: -0.021

Control Structures

Control structures are used to change if and when lines of code in a program are run. The control comes from conditions being true or false.

Root finding with the Newton-Raphson algorithm

The for loop has already been introduced. Two other important control structures are while loops and if statements.

f <- function(x) x^3 + x^2 - 3*x - 3
fp <- function(x) 3*x^2 + 2*x - 3
plot(f, xlim=c(-4,4))

x <- -2
i <- 1
while(abs(f(x)) > 1e-8) {
  x <- x - f(x) / fp(x)
  i <- i + 1
  if(i > 20) {
    print("does not converge")
    break
  }
}
x
[1] -1.732051
f(x)
[1] -4.440892e-16

Worth Mentioning

tidyverse

Wickham’s “Tidy Data” philosophy was concurrent with the development of a suite of R packages useful for data management. This includes ggplot2 and haven as well as many others such as dplyr, stringr, and tidyr.

data.table

data.table is another beneficial package for data manipulation. It provides similar functionality to the data.frame, but works well with large data sets.

LS0tCnRpdGxlOiAiUiB3b3Jrc2hvcCIKYXV0aG9yOiAiQ29sZSBCZWNrIgpvdXRwdXQ6CiAgaHRtbF9ub3RlYm9vazogZGVmYXVsdAotLS0KCiMgTmF2aWdhdGluZyBSU3R1ZGlvCgoqIENyZWF0aW5nIHNjcmlwdHMgKFIgU2NyaXB0LCBSIE5vdGVib29rLCBSIE1hcmtkb3duLCBTaGlueSBXZWIgQXBwKQoqIENyZWF0aW5nIHByb2plY3RzCiogU2Vzc2lvbiAmIFdvcmtpbmcgRGlyZWN0b3J5CiogRW52aXJvbm1lbnQKKiBGaWxlcwoqIFBsb3RzCiogUGFja2FnZXMKKiBIZWxwCgojIEhhcnJlbGwncyBybXMKClNlZSBjaGFwdGVycyAxIGFuZCA5IG9mIFtCaW9zdGF0aXN0aWNzIGZvciBCaW9tZWRpY2FsIFJlc2VhcmNoXVtiYnJdLgoKIyBTZXR1cApgYGB7ciBzZXR1cCxyZXN1bHRzPSdoaWRlJ30KcmVxdWlyZShIbWlzYykKa25pdHJTZXQobGFuZz0nbWFya2Rvd24nLCBoPTQuNSkKb3B0aW9ucyhnclR5cGU9J3Bsb3RseScpCm11IDwtIG1hcmt1cFNwZWNzJGh0bWwKYGBgCgojIFIgUGFja2FnZXMKCkluc3RhbGwgdGhlIHBhY2thZ2VzIGBIbWlzYywgcm1zLCBrbml0ciwgcm1hcmtkb3duYCB0aHJvdWdoIFJTdHVkaW8uCgpQYWNrYWdlcyBjb250YWluIFIgY29kZSBhbmQgbWF5IG9mdGVuIHJlcXVpcmUgdGhlIGluc3RhbGxhdGlvbiBvZiBvdGhlciBwYWNrYWdlcy4gIFdoZW4gSSBpbnN0YWxsIGBIbWlzY2AsIEkgc2VlIHRoYXQgaXRzIGRlcGVuZGVuY3kgYGFjZXBhY2tgIGlzIGF1dG9tYXRpY2FsbHkgaW5zdGFsbGVkOgoKYGBgCkluc3RhbGxpbmcgcGFja2FnZSBpbnRvIOKAmC9ob21lL2JlY2tjYS9SL3g4Nl82NC1wYy1saW51eC1nbnUtbGlicmFyeS8zLjHigJkKKGFzIOKAmGxpYuKAmSBpcyB1bnNwZWNpZmllZCkKYWxzbyBpbnN0YWxsaW5nIHRoZSBkZXBlbmRlbmN5IOKAmGFjZXBhY2vigJkKCnRyeWluZyBVUkwgJ2h0dHA6Ly9kZWJpYW4ubWMudmFuZGVyYmlsdC5lZHUvUi9DUkFOL3NyYy9jb250cmliL2FjZXBhY2tfMS4zLTMuMy50YXIuZ3onCkNvbnRlbnQgdHlwZSAnYXBwbGljYXRpb24veC1nemlwJyBsZW5ndGggMzM1OTAgYnl0ZXMgKDMyIEtiKQpvcGVuZWQgVVJMCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09CmRvd25sb2FkZWQgMzIgS2IKCnRyeWluZyBVUkwgJ2h0dHA6Ly9kZWJpYW4ubWMudmFuZGVyYmlsdC5lZHUvUi9DUkFOL3NyYy9jb250cmliL0htaXNjXzMuMTQtNi50YXIuZ3onCkNvbnRlbnQgdHlwZSAnYXBwbGljYXRpb24veC1nemlwJyBsZW5ndGggNjExMzQ4IGJ5dGVzICg1OTcgS2IpCm9wZW5lZCBVUkwKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KZG93bmxvYWRlZCA1OTcgS2IKCiogaW5zdGFsbGluZyAqc291cmNlKiBwYWNrYWdlIOKAmGFjZXBhY2vigJkgLi4uCi4KLgouCiogRE9ORSAoYWNlcGFjaykKKiBpbnN0YWxsaW5nICpzb3VyY2UqIHBhY2thZ2Ug4oCYSG1pc2PigJkgLi4uCi4KLgouCiogRE9ORSAoSG1pc2MpCmBgYAoKIyMgSW5zdGFsbCBhbmQgdXBkYXRpbmcKCkluc3RhbGwgdGhyb3VnaCBDUkFOIHJlcG9zaXRvcmllcwoKYGBge3IsIGV2YWw9RkFMU0V9CiMgaW5zdGFsbCBzZXZlcmFsIHBhY2thZ2VzCmluc3RhbGwucGFja2FnZXMoYygnSG1pc2MnLCAncm1zJywgJ2tuaXRyJywgJ3JtYXJrZG93bicpKQojIHVwZGF0ZSBhbGwgcGFja2FnZXMKdXBkYXRlLnBhY2thZ2VzKGNoZWNrQnVpbHQ9VFJVRSwgYXNrPUZBTFNFKQpgYGAKCkluc3RhbGwgdGhyb3VnaCBHaXRodWIgcmVwb3NpdG9yaWVzCgpgYGB7ciwgZXZhbD1GQUxTRX0KaW5zdGFsbC5wYWNrYWdlcygnZGV2dG9vbHMnKQpyZXF1aXJlKGRldnRvb2xzKQppbnN0YWxsX2dpdGh1YignaGFycmVsZmUvcm1zJykKYGBgCgojIyBMb2FkaW5nCgpOb3RlIHRoYXQgYGxpYnJhcnlgIGlzIGEgYml0IG9mIG1pc25vbWVyIGFzIFIgdXNlcyBwYWNrYWdlcywgbm90IGxpYnJhcmllcy4gIEZyb20gYSB0ZWNobmljYWwgc3RhbmRwb2ludCwgaXQncyBuaWNlIHRvIHJlY29nbml6ZSB0aGUgZGlzdGluY3Rpb24uICBZb3UgbWF5IHNlZSBgcmVxdWlyZWAgdXNlZCBpbiBpdHMgcGxhY2UuICBJZiB0aGUgcGFja2FnZSBpcyBub3QgaW5zdGFsbGVkLCBgbGlicmFyeWAgd2lsbCB0cmlnZ2VyIGFuIGVycm9yIHdoaWxlIGByZXF1aXJlYCB3aWxsIHJldHVybiBGQUxTRS4gIE9uY2UgbG9hZGVkIHRoZSBmdW5jdGlvbnMgY3JlYXRlZCBpbiB0aGUgcGFja2FnZSBhcmUgYXZhaWxhYmxlIHRvIHlvdXIgUiBzZXNzaW9uLgoKYGBge3IsIGV2YWw9RkFMU0V9CmxpYnJhcnkoSG1pc2MpCnJlcXVpcmUoSG1pc2MpCmBgYAoKIyBGZXRjaGluZyBEYXRhLCBNb2RpZnlpbmcgVmFyaWFibGVzLCBhbmQgUHJpbnRpbmcgRGF0YSBEaWN0aW9uYXJ5CgpUaGUgYGdldEhkYXRhYCBmdW5jdGlvbiBpcyB1c2VkIHRvIGZldGNoIGEgZGF0YXNldCBmcm9tIHRoZSBWYW5kZXJiaWx0IFtEYXRhU2V0c11bZGF0YXNldHNdIHdlYiBzaXRlLiAgYHVwRGF0YWAgaXMgdXNlZCB0bwoKLSBjcmVhdGUgYSBuZXcgdmFyaWFibGUgZnJvbSBhbiBvbGQgb25lCi0gYWRkIGxhYmVscyB0byAyIHZhcmlhYmxlcwotIGFkZCB1bml0cyB0byB0aGUgbmV3IHZhcmlhYmxlCi0gcmVtb3ZlIHRoZSBvbGQgdmFyaWFibGUKLSBhdXRvbWF0aWNhbGx5IG1vdmUgdW5pdHMgb2YgbWVhc3VyZW1lbnRzIGZyb20gcGFyZW50aGV0aWNhbCBleHByZXNzaW9ucyBpbiBsYWJlbHMgdG8gc2VwYXJhdGUgYHVuaXRzYCBhdHRyaWJ1dGVkIHVzZWQgYnkgYEhtaXNjYCBhbmQgYHJtc2AgZnVuY3Rpb25zIGZvciB0YWJsZSBtYWtpbmcgYW5kIGdyYXBoaWNzCgpgY29udGVudHNgIGlzIHVzZWQgdG8gcHJpbnQgYSBkYXRhIGRpY3Rpb25hcnksIHJ1biB0aHJvdWdoIGFuIGBodG1sYCBtZXRob2QgZm9yIG5pY2VyIG91dHB1dC4KCmBgYHtyIG1ldGFkYXRhLHJlc3VsdHM9J2FzaXMnfQpnZXRIZGF0YShwYmMpCnBiYyA8LSB1cERhdGEocGJjLAogICAgICAgICAgICAgIGZ1LnlycyA9IGZ1LmRheXMgLyAzNjUuMjUsCiAgICAgICAgICAgICAgbGFiZWxzID0gYyhmdS55cnMgPSAnRm9sbG93LXVwIFRpbWUnLAogICAgICAgICAgICAgICAgICAgICAgICAgc3RhdHVzID0gJ0RlYXRoIG9yIExpdmVyIFRyYW5zcGxhbnRhdGlvbicpLAogICAgICAgICAgICAgIHVuaXRzID0gYyhmdS55cnMgPSAneWVhcicpLAogICAgICAgICAgICAgIGRyb3AgID0gJ2Z1LmRheXMnLAogICAgICAgICAgICAgIG1vdmVVbml0cz1UUlVFLCBodG1sPVRSVUUpCmh0bWwoY29udGVudHMocGJjKSwgbWF4bGV2ZWxzPTEwLCBsZXZlbFR5cGU9J3RhYmxlJykKYGBgCgojIERlc2NyaXB0aXZlIFN0YXRpc3RpY3MgV2l0aG91dCBTdHJhdGlmaWNhdGlvbgoKYGBge3IgZGVzY3JpYmUscmVzdWx0cz0nYXNpcyd9CiMgZGlkIGhhdmUgcmVzdWx0cz0nYXNpcycgYWJvdmUKZCA8LSBkZXNjcmliZShwYmMpCmh0bWwoZCwgc2l6ZT04MCwgc2Nyb2xsPVRSVUUpCnBsb3QoZCkKYGBgCgojIFN0cmF0aWZpZWQgRGVzY3JpcHRpdmUgU3RhdGlzdGljcwoKUHJvZHVjZSBzdHJhdGlmaWVkIHF1YW50aWxlcywgbWVhbnMvU0QsIGFuZCBwcm9wb3J0aW9ucyBieSB0cmVhdG1lbnQgZ3JvdXAuICBQbG90IHRoZSByZXN1bHRzIGJlZm9yZSByZW5kZXJpbmcgYXMgYW4gYWR2YW5jZWQgaHRtbCB0YWJsZToKCi0gY2F0ZWdvcmljYWwgdmFyaWFibGVzOiBhIHNpbmdsZSBkb3QgY2hhcnQKLSBjb250aW51b3VzIHZhcmlhYmxlczogYSBzZXJpZXMgb2YgZXh0ZW5kZWQgYm94IHBsb3RzCgpgYGB7ciBzdW1tYXJ5TSxyZXN1bHRzPSdhc2lzJ30KcyA8LSBzdW1tYXJ5TShiaWxpICsgYWxidW1pbiArIHN0YWdlICsgcHJvdGltZSArIHNleCArIGFnZSArIHNwaWRlcnMgKwogICAgICAgICAgICAgIGFsay5waG9zICsgc2dvdCArIGNob2wgfiBkcnVnLCBkYXRhPXBiYywKCQkJCQkJCW92ZXJhbGw9RkFMU0UsIHRlc3Q9VFJVRSkKcGxvdChzLCB3aGljaD0nY2F0ZWdvcmljYWwnKQpwbG90KHMsIHdoaWNoPSdjb250aW51b3VzJywgdmFycz0xOjQpCnBsb3Qocywgd2hpY2g9J2NvbnRpbnVvdXMnLCB2YXJzPTU6NykKYGBgCgpgYGB7ciBzdW1tYXJ5TTR9Cmh0bWwocywgY2FwdGlvbj0nQmFzZWxpbmUgY2hhcmFjdGVyaXN0aWNzIGJ5IHJhbmRvbWl6ZWQgdHJlYXRtZW50JywKICAgICBleGNsdWRlMT1UUlVFLCBucGN0PSdib3RoJywgZGlnaXRzPTMsCiAgICAgcHJtc2Q9VFJVRSwgYnJtc2Q9VFJVRSwgbXNkc2l6ZT1tdSRzbWFsbGVyMikKYGBgCgojIFNwaWtlIEhpc3RvZ3JhbQoKYGBge3IgaGlzdGJveCxyZXN1bHRzPSdhc2lzJ30KcCA8LSB3aXRoKHBiYywgaGlzdGJveHAoeD1zZ290LCBncm91cD1kcnVnLCBzZD1UUlVFKSkKcApgYGAKIyBEYXRhIFZpc3VhbGl6YXRpb24KClRoZSB2ZXJ5IGxvdyBiaXJ0aHdlaWdodCBkYXRhIHNldCBjb250YWlucyBkYXRhIG9uIDY3MSBpbmZhbnRzIGJvcm4gd2l0aCBhIGJpcnRoIHdlaWdodCBvZiB1bmRlciAxNjAwIGdyYW1zLiAgV2UnbGwgcGxvdCBnZXN0YXRpb25hbCBhZ2UgYnkgYmlydGh3ZWlnaHQgdXNpbmcgdGhyZWUgZ3JhcGhpY3Mgc3lzdGVtczogYmFzZSBncmFwaGljcywgZ2dwbG90LCBhbmQgcGxvdGx5LgoKYGBge3J9CmdldEhkYXRhKHZsYncpCiMgcmVtb3ZlIG1pc3NpbmcgdmFsdWVzCnZsYncgPC0gdmxid1tjb21wbGV0ZS5jYXNlcyh2bGJ3WyxjKCdzZXgnLCdkZWFkJywnZ2VzdCcsJ2J3dCcpXSksXQpgYGAKCiMjIEJhc2UKCkJ1aWxkIGVhY2ggZWxlbWVudCBpbnRvIHlvdXIgcGxvdC4KCmBgYHtyLCBmaWcud2lkdGggPSA2fQpncnBzIDwtIHNwbGl0KHZsYndbLGMoJ2dlc3QnLCdid3QnKV0sIHZsYndbLGMoJ3NleCcsJ2RlYWQnKV0pCnBsb3QoYygyMiw0MCksIGMoNDAwLDE2MDApLCB0eXBlPSduJywgeGxhYj0nR2VzdGF0aW9uYWwgQWdlJywgeWxhYj0nQmlydGggV2VpZ2h0IChncmFtcyknLCBheGVzPUZBTFNFKQpheGlzKDEsIGF0PWMoMjIsMjgsMzQsNDApLCBsYWJlbHM9YygyMiwyOCwzNCw0MCkpCmF4aXMoMiwgYXQ9c2VxKDQwMCwxNjAwLGJ5PTQwMCksIGxhYmVscz1zZXEoNDAwLDE2MDAsYnk9NDAwKSkKcG9pbnRzKGdycHNbWydmZW1hbGUuMCddXSwgY29sPSdibGFjaycsIHBjaD0xKQpwb2ludHMoZ3Jwc1tbJ2ZlbWFsZS4xJ11dWywnZ2VzdCddLCBncnBzW1snZmVtYWxlLjEnXV1bLCdid3QnXSwgY29sPSdibGFjaycsIHBjaD0wKQpwb2ludHMoaml0dGVyKGdycHNbWydtYWxlLjAnXV1bLCdnZXN0J10sIDIpLCBncnBzW1snbWFsZS4wJ11dWywnYnd0J10sIGNvbD0nZ3JheScsIHBjaD00KQpwb2ludHMoZ3Jwc1tbJ21hbGUuMCddXVssJ2J3dCddIH4gaml0dGVyKGdycHNbWydtYWxlLjAnXV1bLCdnZXN0J10sIDIpLCBjb2w9J2dyYXknLCBwY2g9MykKbGVnZW5kKHg9MzgsIHk9MTAwMCwgbGVnZW5kPWMoJ0Y6MCcsJ0Y6MScsJ006MCcsJ006MScpLCBjb2w9YygnYmxhY2snLCdibGFjaycsJ2dyYXknLCdncmF5JyksIHBjaD1jKDEsMCw0LDMpKQpgYGAKCgojIyBnZ3Bsb3QyCgpHaXZlbiBhIGRhdGEgc2V0LCBjaG9vc2UgdGhlIGFlc3RoZXRpYyBtYXBwaW5nIGFuZCBnZW9tZXRyeSBsYXllci4KCmBgYHtyfQpwIDwtIGdncGxvdChkYXRhPXZsYncpICsgYWVzKHg9Z2VzdCwgeT1id3QsIGNvbG9yPXNleCwgc2hhcGU9YXMuZmFjdG9yKGRlYWQpKSArIGdlb21fcG9pbnQoKQpwCmBgYAoKYGBge3J9CnAgPC0gcCArIGdlb21faml0dGVyKCkgKyBzY2FsZV94X2NvbnRpbnVvdXMoKSArIHNjYWxlX3lfY29udGludW91cygpCnAKYGBgCgojIyBQbG90bHkKCkFkZCBpbnRlcmFjdGl2ZSBncmFwaGljcywgd2hpY2ggaXMgdHJpdmlhbCB3aGVuIGFsc28gdXNpbmcgYGdncGxvdGAuCgpgYGB7cn0KcmVxdWlyZShwbG90bHkpCmdncGxvdGx5KHApCmBgYAoKYGBge3J9CnBsb3RfbHkodHlwZT0iYm94IikgJT4lCiAgYWRkX2JveHBsb3QoeSA9IGdycHNbWydmZW1hbGUuMCddXVssJ2J3dCddLCBuYW1lPSdGOjAnKSAlPiUKICBhZGRfYm94cGxvdCh5ID0gZ3Jwc1tbJ2ZlbWFsZS4xJ11dWywnYnd0J10sIG5hbWU9J0Y6MScpICU+JQogIGFkZF9ib3hwbG90KHkgPSBncnBzW1snbWFsZS4wJ11dWywnYnd0J10sIG5hbWU9J006MCcpICU+JQogIGFkZF9ib3hwbG90KHkgPSBncnBzW1snbWFsZS4xJ11dWywnYnd0J10sIG5hbWU9J006MScpCmBgYAoKIyBDYXJkaW92YXNjdWxhciByaXNrIGZhY3RvciBkYXRhCgpgYGB7cn0KZ2V0SGRhdGEoZGlhYmV0ZXMpCmh0bWwoY29udGVudHMoZGlhYmV0ZXMpLCBsZXZlbFR5cGU9J3RhYmxlJykKYGBgCgojIEZvcm11bGFzCgpUaGUgdGlsZGUgaXMgdXNlZCB0byBjcmVhdGUgYSBtb2RlbCBmb3JtdWxhLCB3aGljaCBjb25zaXN0cyBvZiBhIGxlZnQtaGFuZCBzaWRlIGFuZCByaWdodC1oYW5kIHNpZGUuICBNYW55IFIgZnVuY3Rpb25zIHV0aWxpemUgZm9ybXVsYXMsIHN1Y2ggYXMgcGxvdHRpbmcgZnVuY3Rpb25zIGFuZCBtb2RlbC1maXR0aW5nIGZ1bmN0aW9ucy4gIFRoZSBsZWZ0LWhhbmQgc2lkZSBjb25zaXN0cyBvZiB0aGUgcmVzcG9uc2UgdmFyaWFibGUsIHdoaWxlIHRoZSByaWdodC1oYW5kIHNpZGUgbWF5IGNvbnRhaW4gc2V2ZXJhbCB0ZXJtcy4gIFlvdSBtYXkgc2VlIHRoZSBmb2xsb3dpbmcgb3BlcmF0b3JzIHdpdGhpbiBhIGZvcm11bGEuCgoqIHkgfiBhICsgYiwgYCtgIGluZGljYXRlcyB0byBpbmNsdWRlIGJvdGggYSBhbmQgYiBhcyB0ZXJtcwoqIHkgfiBhOmIsIGA6YCBpbmRpY2F0ZXMgdGhlIGludGVyYWN0aW9uIG9mIGEgYW5kIGIKKiB5IH4gYSpiLCBlcXVpdmFsZW50IHRvIHkgfiBhK2IrYTpiCiogeSB+IChhK2IpXjIsIGVxdWl2YWxlbnQgdG8geSB+IChhK2IpKihhK2IpCiogeSB+IGEgKyBJKGIrYyksIGBJYCBpbmRpY2F0ZXMgdG8gdXNlIGArYCBpbiB0aGUgYXJpdGhtZXRpYyBzZW5zZQoKU2VlID9mb3JtdWxhIGZvciBtb3JlIGV4YW1wbGVzLgoKIyBNb2RlbHMKClRoZSBtb3N0IHNpbXBsZSBtb2RlbC1maXR0aW5nIGZ1bmN0aW9uIGlzIGBsbWAsIHdoaWNoIGlzIHVzZWQgdG8gZml0IGxpbmVhciBtb2RlbHMuICBJdCdzIHByaW1hcnkgYXJndW1lbnQgaXMgYSBmb3JtdWxhLiAgVXNpbmcgdGhlIGRpYWJldGVzIGRhdGEgc2V0LCB3ZSBjYW4gZml0IHdhaXN0IHNpemUgYnkgd2VpZ2h0LgoKYGBge3J9CihtIDwtIGxtKHdhaXN0IH4gd2VpZ2h0LCBkYXRhPWRpYWJldGVzKSkKYGBgCgpUaGlzIGNyZWF0ZXMgYW4gYGxtYCBvYmplY3QsIGFuZCBzZXZlcmFsIGZ1bmN0aW9ucyBjYW4gYmUgdXNlZCBvbiBtb2RlbCBvYmplY3RzLiAgVGhlIGludGVybmFsIHN0cnVjdHVyZSBvZiBhIG1vZGVsIG9iamVjdCBpcyBhIGxpc3QgLSBpdHMgZWxlbWVudHMgbWF5IGJlIGFjY2Vzc2VkIGp1c3QgbGlrZSBhIGxpc3QuCgoqIGNvZWYKKiBmaXR0ZWQKKiBwcmVkaWN0CiogcmVzaWR1YWxzCiogdmNvdgoqIHN1bW1hcnkKKiBhbm92YQoKYGBge3J9Cm5hbWVzKG0pCm0kY29lZmZpY2llbnRzCmNvZWYobSkKaGVhZChmaXR0ZWQobSkpCnByZWRpY3QobSwgZGF0YS5mcmFtZSh3ZWlnaHQ9YygxNTAsIDIwMCwgMjUwKSkpCmhlYWQocmVzaWR1YWxzKG0pKQp2Y292KG0pCnN1bW1hcnkobSkKY29lZihzdW1tYXJ5KG0pKQphbm92YShtKQpgYGAKCiMjIFZpc3VhbGl6YXRpb24KCkNyZWF0ZSBhIHNjYXR0ZXJwbG90IG9mIHdlaWdodCBhbmQgd2Fpc3Qgc2l6ZS4KCmBgYHtyfQpwIDwtIHFwbG90KHdlaWdodCwgd2Fpc3QsIGRhdGE9ZGlhYmV0ZXMpCnAgKyBnZW9tX3Ntb290aChtZXRob2Q9ImxtIikKYGBgCgpSZW1vdmUgbWlzc2luZyB2YWx1ZXMgYW5kIGZpdCB3aXRoIExPRVNTIGN1cnZlLgoKYGBge3J9CmRpYWIgPC0gZGlhYmV0ZXNbY29tcGxldGUuY2FzZXMoZGlhYmV0ZXNbLGMoJ3dhaXN0Jywnd2VpZ2h0JyldKSxdCnAgPC0gcXBsb3Qod2VpZ2h0LCB3YWlzdCwgZGF0YT1kaWFiKQpwICsgZ2VvbV9zbW9vdGgobWV0aG9kPSJsb2VzcyIpCmBgYAoKIyBEYXRhIE1hbmlwdWxhdGlvbgoKVGhlIERvbWluaWNhbiBSZXB1YmxpYyBIeXBlcnRlbnNpb24gZGF0YXNldCBjb250YWlucyBkYXRhIG9uIGdlbmRlciwgYWdlLCBhbmQgc3lzdG9saWMgYW5kIGRpYXN0b2xpYyBibG9vZCBwcmVzc3VyZSBmcm9tIHNldmVyYWwgdmlsbGFnZXMgaW4gdGhlIERSLgoKYGBge3J9CmdldEhkYXRhKERvbWluaWNhbkhUTikKaHRtbChkZXNjcmliZShEb21pbmljYW5IVE4pKQpgYGAKCiMjIEFkZGluZyB2YXJpYWJsZXMgb3IgdHJhbnNmb3JtYXRpb25zCgpgYGB7cn0KRG9taW5pY2FuSFROWywnbWFwJ10gPC0gKERvbWluaWNhbkhUTlssJ3NicCddICsgRG9taW5pY2FuSFROWywnZGJwJ10gKiAyKSAvIDMKRG9taW5pY2FuSFROWywnbWFsZSddIDwtIGFzLm51bWVyaWMoRG9taW5pY2FuSFROWywnZ2VuZGVyJ10gPT0gJ01hbGUnKQpEb21pbmljYW5IVE5bMTo1LF0KYGBgCgpgYGB7cn0KcWFnZSA8LSBxdWFudGlsZShEb21pbmljYW5IVE5bLCdhZ2UnXSkKcWFnZVsxXSA8LSBxYWdlWzFdLTEKRG9taW5pY2FuSFROWywnYWdlR3JwJ10gPC0gY3V0KERvbWluaWNhbkhUTlssJ2FnZSddLCBicmVha3M9cWFnZSkKYGBgCgojIyBGaWx0ZXIKCmBgYHtyfQpucm93KERvbWluaWNhbkhUTikKbnJvdyhEb21pbmljYW5IVE5bRG9taW5pY2FuSFROWywnZ2VuZGVyJ10gPT0gIk1hbGUiLF0pCndoaWNoKERvbWluaWNhbkhUTlssJ3ZpbGxhZ2UnXSAlaW4lIGMoJ0Nhcm1vbmEnLCdTYW4gQW50b25pbycpKQpjb2pNZW4gPC0gRG9taW5pY2FuSFROW0RvbWluaWNhbkhUTlssJ3ZpbGxhZ2UnXSA9PSAnQ29qb2JhbCcgJiBEb21pbmljYW5IVE5bLCdnZW5kZXInXSA9PSAiTWFsZSIsXQpjb2pNZW4KYGBgCgojIyBTb3J0CgpgYGB7cn0KY29qTWVuW29yZGVyKGNvak1lblssJ2FnZSddLCBkZWNyZWFzaW5nPVRSVUUpLF0KYGBgCgojIyBBZ2dyZWdhdGUgYW5kIGNvdW50cwoKYGBge3J9CnRhYmxlKERvbWluaWNhbkhUTlssJ2dlbmRlciddKQpgYGAKCmBgYHtyfQphZGRtYXJnaW5zKHdpdGgoRG9taW5pY2FuSFROLCB0YWJsZSh2aWxsYWdlLCBnZW5kZXIpKSkKYGBgCgpgYGB7cn0Kd2l0aChEb21pbmljYW5IVE4sIHRhcHBseShhZ2UsIHZpbGxhZ2UsIG1lYW4pKQpgYGAKCmBgYHtyfQphZ2dyZWdhdGUoc2JwIH4gZ2VuZGVyLCBkYXRhPURvbWluaWNhbkhUTiwgRlVOPW1lYW4pCmBgYAoKYGBge3J9CmFnZ3JlZ2F0ZShhZ2UgfiB2aWxsYWdlICsgZ2VuZGVyLCBEb21pbmljYW5IVE4sIG1lZGlhbikKYGBgCgojIFJvc25lcidzIGxlYWQgZGF0YQoKVGhlIGBsZWFkYCBleHBvc3VyZSBkYXRhc2V0IHdhcyBjb2xsZWN0ZWQgdG8gc3R1ZHkgdGhlIHdlbGwtYmVpbmcgb2YgY2hpbGRlbiB3aG8gbGl2ZWQgbmVhciBhIGxlYWQgc21lbHRpbmcgcGxhbnQuICBUaGUgZm9sbG93aW5nIGFuYWx5c2lzIGNvbnNpZGVycyB0aGUgbGVhZCBleHBvc3VyZSBsZXZlbHMgaW4gMTk3MiBhbmQgMTk3MywgYXMgd2VsbCBhcyBhZ2UgYW5kIHRoZSBmaW5nZXItd3Jpc3QgdGFwcGluZyBzY29yZSBgbWF4Znd0YC4KCmBgYHtyIGNvbnRlbnRzLHJlc3VsdHM9J2FzaXMnfQpnZXRIZGF0YShsZWFkKQpsZWFkIDwtIHVwRGF0YShsZWFkLAogICAgICAga2VlcCA9IGMoJ2xkNzInLCdsZDczJywnYWdlJywnbWF4Znd0JyksCiAgICAgICBsYWJlbHMgPSBjKGFnZSA9ICdBZ2UnKSwKICAgICAgIHVuaXRzID0gYyhhZ2U9J3llYXJzJywgbGQ3Mj0nbWcvMTAwKm1sJywgbGQ3Mz0nbWcvMTAwKm1sJykpCmh0bWwoY29udGVudHMobGVhZCksIG1heGxldmVscz0xMCwgbGV2ZWxUeXBlPSd0YWJsZScpCmBgYAoKYGBge3J9Cmh0bWwoZGVzY3JpYmUobGVhZCkpCmBgYAoKIyBPcmRpbmFyeSBsZWFzdCBzcXVhcmVzIHJlZ3Jlc3Npb24KClRoZSBgcm1zYCBwYWNrYWdlIGluY2x1ZGVzIHRoZSBgb2xzYCBmaXR0aW5nIGZ1bmN0aW9uLiAgVXNlIGBkYXRhZGlzdGAgdG8gY29tcHV0ZSBzdW1tYXJpZXMgb2YgdGhlIGRpc3RyaWJ1dGlvbmFsIGNoYXJhdGVyaXN0aWNzIG9mIHRoZSBwcmVkaWN0b3JzIC0gb3Igc2ltcGx5IGdpdmUgaXQgYW4gZW50aXJlIGRhdGEgZnJhbWUuICBgZGF0YWRpc3RgIG11c3QgYmUgcmUtcnVuIGlmIHlvdSBhZGQgYSBuZXcgcHJlZGljdG9yIG9yIHJlY29kZSBhbiBvbGQgb25lLgoKYGBge3J9CnJlcXVpcmUocm1zKQpkZCA8LSBkYXRhZGlzdChsZWFkKQpvcHRpb25zKGRhdGFkaXN0PSdkZCcpCmYgPC0gb2xzKG1heGZ3dCB+IGFnZSArIGxkNzIgKyBsZDczLCBkYXRhPWxlYWQpCmYKYGBgCgpgcm1zYCBwcm92aWRlcyBtYW55IG1ldGhvZHMgdG8gd29yayB3aXRoIHRoZSBgb2xzYCBmaXQgb2JqZWN0LgoKKiBjb2VmCiogZml0dGVkCiogcHJlZGljdAoqIHJlc2lkCiogYW5vdmEKKiBzdW1tYXJ5CiogUHJlZGljdAoqIGdncGxvdAoqIEZ1bmN0aW9uCiogbm9tb2dyYW0KCiMjIENvZWZmaWNpZW50IEVzdGltYXRlcwoKYGBge3J9CmNvZWYoZikKYGBgCgojIyBEZWZpbmUgUiBmdW5jdGlvbiByZXByZXNlbnRpbmcgZml0dGVkIG1vZGVsCgpUaGUgZGVmYXVsdCB2YWx1ZXMgZm9yIGZ1bmN0aW9uIGFyZ3VtZW50cyBhcmUgbWVkaWFucy4KCmBgYHtyfQpnIDwtIEZ1bmN0aW9uKGYpCmcKZyhhZ2U9MTAsIGxkNzI9MjEsIGxkNzM9YygyMSwgNDcpKQpgYGAKCiMjIFByZWRpY3RlZCB2YWx1ZXMgd2l0aCBzdGFuZGFyZCBlcnJvcnMKCmBgYHtyfQpwcmVkaWN0KGYsIGRhdGEuZnJhbWUoYWdlPTEwLCBsZDcyPTIxLCBsZDczPWMoMjEsIDQ3KSksIHNlLmZpdD1UUlVFKQpgYGAKCiMjIFJlc2lkdWFscwoKYGBge3J9CnIgPC0gcmVzaWQoZikKcGFyKG1mcm93PWMoMiwyKSkKcGxvdChmaXR0ZWQoZiksIHIpOyBhYmxpbmUoaD0wKQp3aXRoKGxlYWQsIHBsb3QoYWdlLCByKSk7IGFibGluZShoPTApCndpdGgobGVhZCwgcGxvdChsZDczLCByKSk7IGFibGluZShoPTApCiMgcS1xIHBsb3QgdG8gY2hlY2sgbm9ybWFsaXR5CnFxbm9ybShyKQpgYGAKCiMjIFBsb3R0aW5nIHBhcnRpYWwgZWZmZWN0cwoKYFByZWRpY3RgIGFuZCBgcHBsb3RgIG1ha2Ugb25lIHBsb3QgZm9yIGVhY2ggcHJlZGljdG9yLiAgUHJlZGljdG9ycyBub3Qgc2hvd24gaW4gcGxvdCBhcmUgc2V0IHRvIGNvbnN0YW50cyAoY29udGludW91czptZWRpYW4sIGNhdGVnb3JpY2FsOm1vZGUpLiAgMC45NSBwb2ludHdpc2UgY29uZmlkZW5jZSBsaW1pdHMgZm9yICRcaGF0e0V9KHl8eCkkIGFyZSBzaG93bjsgdXNlIGBjb25mLmludD1GQUxTRWAgdG8gc3VwcHJlc3MgQ0xzLgoKYGBge3J9CnBsb3RwKFByZWRpY3QoZikpCmBgYAoKU3BlY2lmeSB3aGljaCBwcmVkaWN0b3JzIGFyZSBwbG90dGVkLgoKYGBge3J9CnBsb3RwKFByZWRpY3QoZiwgYWdlKSkKYGBgCgpgYGB7cn0KcGxvdHAoUHJlZGljdChmLCBhZ2U9MzoxNSkpCmBgYAoKYGBge3J9CnBsb3RwKFByZWRpY3QoZiwgYWdlPXNlcSgzLDE2LGxlbmd0aD0xNTApKSkKYGBgCgpPYnRhaW4gY29uZmlkZW5jZSBsaW1pdHMgZm9yICRcaGF0e3l9JC4KCmBgYHtyfQpwbG90cChQcmVkaWN0KGYsIGFnZSwgY29uZi50eXBlPSdpbmRpdmlkdWFsJykpCmBgYAoKQ29tYmluZSBwbG90cy4KCmBgYHtyfQpwMSA8LSBQcmVkaWN0KGYsIGFnZSwgY29uZi5pbnQ9MC45OSwgY29uZi50eXBlPSdpbmRpdmlkdWFsJykKcDIgPC0gUHJlZGljdChmLCBhZ2UsIGNvbmYuaW50PTAuOTksIGNvbmYudHlwZT0nbWVhbicpCnAgPC0gcmJpbmQoSW5kaXZpZHVhbD1wMSwgTWVhbj1wMikKZ2dwbG90KHApCmBgYAoKYGBge3J9CnBsb3RwKFByZWRpY3QoZiwgbGQ3MywgYWdlPTMpKQpgYGAKCmBgYHtyfQpnZ3Bsb3QoUHJlZGljdChmLCBsZDczLCBhZ2U9YygzLDkpKSkKYGBgCgozLWQgc3VyZmFjZSBmb3IgdHdvIGNvbnRpbnVvdXMgcHJlZGljdG9ycyBhZ2FpbnN0ICRcaGF0e3l9JC4KCmBgYHtyLCBmaWcud2lkdGg9Nn0KYnBsb3QoUHJlZGljdChmLCBsZDcyLCBsZDczKSkKYGBgCgojIyBOb21vZ3JhbXMKCmBgYHtyLCBmaWcuaGVpZ2h0PTYsIGZpZy53aWR0aD02fQpwbG90KG5vbW9ncmFtKGYpKQpgYGAKCiMjIFBvaW50IEVzdGltYXRlcyBmb3IgUGFydGlhbCBFZmZlY3RzCgpgYGB7cn0Kc3VtbWFyeShmKQpgYGAKCkFkanVzdCBgYWdlYCB0byA1LCB3aGljaCBoYXMgbm8gZWZmZWN0IGFzIHRoZSBtb2RlbCBkb2VzIG5vdCBpbmNsdWRlIGFueSBpbnRlcmFjdGlvbnMuCgpgYGB7cn0Kc3VtbWFyeShmLCBhZ2U9NSkKYGBgCgpFZmZlY3Qgb2YgY2hhbmdpbmcgYGxkNzNgIGZyb20gMjAgdG8gNDAuCgpgYGB7cn0Kc3VtbWFyeShmLCBsZDczPWMoMjAsNDApKQpgYGAKCklmIGEgcHJlZGljdG9yIGhhcyBhIGxpbmVhciBlZmZlY3QsIGEgb25lLXVuaXQgY2hhbmdlIGNhbiBiZSB1c2VkIHRvIGdldCB0aGUgY29uZmlkZW5jZSBpbnRlcnZhbCBvZiBpdHMgc2xvcGUuCgpgYGB7cn0Kc3VtbWFyeShmLCBhZ2U9NTo2KQpgYGAKClRoZXJlIGlzIGFsc28gYSBgcGxvdGAgbWV0aG9kIGZvciBgc3VtbWFyeWAgcmVzdWx0cy4KCmBgYHtyfQpwbG90KHN1bW1hcnkoZikpCmBgYAoKIyMgR2V0dGluZyBQcmVkaWN0ZWQgVmFsdWVzCgpHaXZlIGBwcmVkaWN0YCBhIGZpdCBvYmplY3QgYW5kIGBkYXRhLmZyYW1lYC4KCmBgYHtyfQpwcmVkaWN0KGYsIGRhdGEuZnJhbWUoYWdlPTMsIGxkNzI9MjEsIGxkNzM9MjEpKQpgYGAKCmBgYHtyfQpwcmVkaWN0KGYsIGRhdGEuZnJhbWUoYWdlPWMoMywxMCksIGxkNzI9MjEsIGxkNzM9YygyMSw0NykpKQpgYGAKCmBgYHtyfQpuZXdkYXQgPC0gZXhwYW5kLmdyaWQoYWdlPWMoNCw4KSwgbGQ3Mj1jKDIxLCA0NyksIGxkNzM9YygyMSwgNDcpKQpuZXdkYXQKYGBgCgpgYGB7cn0KcHJlZGljdChmLCBuZXdkYXQpCmBgYAoKSW5jbHVkZSBjb25maWRlbmNlIGxldmVscy4KCmBgYHtyfQpwcmVkaWN0KGYsIG5ld2RhdCwgY29uZi5pbnQ9MC45NSkKcHJlZGljdChmLCBuZXdkYXQsIGNvbmYuaW50PTAuOTUsIGNvbmYudHlwZT0naW5kaXZpZHVhbCcpCmBgYAoKQnJ1dGUtZm9yY2UgcHJlZGljdGVkIHZhbHVlcy4KCmBgYHtyfQpiIDwtIGNvZWYoZikKYlsxXSArIGJbMl0qMyArIGJbM10qMjEgKyBiWzRdKjIxCmBgYAoKUHJlZGljdGVkIHZhbHVlcyB3aXRoIGBGdW5jdGlvbmAuCgpgYGB7cn0KIyBnIDwtIEZ1bmN0aW9uKGYpCmcoYWdlID0gYygzLDgpLCBsZDcyID0gMjEsIGxkNzMgPSAyMSkKZyhhZ2UgPSAzKQpgYGAKCiMjIEFOT1ZBCgpVc2UgYGFub3ZhYCB0byBnZXQgYWxsIHRvdGFsIGVmZmVjdHMgYW5kIGluZGl2aWR1YWwgcGFydGlhbCBlZmZlY3RzLgoKYGBge3J9CmFuIDwtIGFub3ZhKGYpCmFuCmBgYAoKSW5jbHVkZSBjb3JyZXNwb25kaW5nIHZhcmlhYmxlIG5hbWVzLgoKYGBge3J9CnByaW50KGFuLCAnbmFtZXMnKQpwcmludChhbiwgJ3N1YnNjcmlwdHMnKQpwcmludChhbiwgJ2RvdHMnKQpgYGAKCkNvbWJpbmVkIHBhcnRpYWwgZWZmZWN0cy4KCmBgYHtyfQphbm92YShmLCBsZDcyLCBsZDczKQpgYGAKCiMgSW1wb3J0aW5nIERhdGEKCiMjIFRleHQgZmlsZXMKCiogcmVhZC50YWJsZSAtIHdvcmsgd2l0aCBkYXRhIGluIHRhYmxlIGZvcm1hdAoqIHJlYWQuY3N2IC0gQ1NWIGZpbGVzCiogcmVhZC5kZWxpbSAtIHRhYi1kZWxpbWl0ZWQgZmlsZXMKKiBzY2FuIC0gbW9yZSBnZW5lcmFsIGZ1bmN0aW9uIGZvciByZWFkaW5nIGluIGRhdGEKCiMjIEJpbmFyeSByZXByZXNlbnRhdGlvbiBvZiBSIG9iamVjdHMKCkNvbXByZXNzZWQgZGF0YSBhbmQgbG9hZHMgZmFzdGVyLgoKKiBSRGF0YQoqIGBmZWF0aGVyYCBwYWNrYWdlCgojIyBEYXRhYmFzZSBjb25uZWN0aW9ucwoKQWxsb3dzIGRhdGEgcXVlcmllcy4KCiogYFJPREJDYCBwYWNrYWdlCiogYFJTUUxpdGVgIHBhY2thZ2UKCiMjIE90aGVyIHN0YXRpc3RpY2FsIHBhY2thZ2VzCgoqIGBIbWlzY2AgcGFja2FnZTogYHNhcy5nZXRgLCBgc2FzeHBvcnQuZ2V0YAoqIGBoYXZlbmAgcGFja2FnZTogYHJlYWRfc2FzYCwgYHJlYWRfc2F2YCwgYHJlYWRfZHRhYAoqIGBmb3JlaWduYCBwYWNrYWdlCiogU3RhdC9UcmFuc2ZlciAtIG5vdCBmcmVlIG5vciBvcGVuIHNvdXJjZQoKIyBEYXRhIENsZWFuaW5nCgpBbiBSIGBkYXRhLmZyYW1lYCBjb25zaXN0cyBvZiBhIGNvbGxlY3Rpb24gb2YgdmFsdWVzLiAgSW4gYSB3ZWxsLXN0cnVjdHVyZWQgZGF0YSBzZXQsIGVhY2ggdmFsdWUgaXMgYXNzb2NpYXRlZCB3aXRoIGEgdmFyaWFibGUgKGNvbHVtbikgYW5kIG9ic2VydmF0aW9uIChyb3cpLiAgRGF0YSBzZXRzIG9mdGVuIG5lZWQgdG8gYmUgbWFuaXB1bGF0ZWQgdG8gYmVjb21lIHdlbGwtc3RydWN0dXJlZC4KCkhhZGxleSBXaWNraGFtJ3MgW1RpZHkgRGF0YV1bdGlkeV0gcHJvdmlkZXMgYW4gZXhjZWxsZW50IG92ZXJ2aWV3IG9uIGhvdyB0byBjbGVhbiBtZXNzeSBkYXRhIHNldHMuCgojIyBDb2x1bW4gaGVhZGVyIGNvbnRhaW5zIHZhbHVlcwoKYGBge3J9CiMgcG9saXRpY3MgYW5kIHJlbGlnaW9uCnNlbmF0ZVJlbGlnaW9uIDwtIGRhdGEuZnJhbWUocmVsaWdpb249YygnUHJvdGVzdGFudCcsJ0NhdGhvbGljJywnSmV3aXNoJywnTW9ybW9uJywnQnVkZGhpc3QnLCJEb24ndCBLbm93L1JlZnVzZWQiKSwKICAgICAgICAgICAgICAgICAgICAgICAgICBEZW1vY3JhdHM9YygyMCwxNSw4LDEsMSwzKSwKICAgICAgICAgICAgICAgICAgICAgICAgICBSZXB1YmxpY2Fucz1jKDM4LDksMCw1LDAsMCkpCmBgYAoKYGBge3J9CnNlblJlbCA8LSBjYmluZChzZW5hdGVSZWxpZ2lvblssJ3JlbGlnaW9uJ10sIHN0YWNrKHNlbmF0ZVJlbGlnaW9uWyxjKCdEZW1vY3JhdHMnLCdSZXB1YmxpY2FucycpXSkpCm5hbWVzKHNlblJlbCkgPC0gYygncmVsaWdpb24nLCdjb3VudCcsJ3BhcnR5JykKc2VuUmVsCmBgYAoKIyMgTXVsdGlwbGUgdmFyaWFibGVzIGluIGEgY29sdW1uCgpgYGB7cn0KcGV0cyA8LSBkYXRhLmZyYW1lKGNvdW50eT1jKCdEYXZpZHNvbicsJ1J1dGhlcmZvcmQnLCdDYW5ub24nKSwKICAgICAgICAgICAgICAgICAgIG1hbGUuZG9nPWMoNTAsMTUwLDcwKSwgZmVtYWxlLmRvZz1jKDYwLDE1MCw3MCksCiAgICAgICAgICAgICAgICAgICBtYWxlLmNhdD1jKDMwLDEwMCw1MCksIGZlbWFsZS5jYXQ9YygzMCw3MCw0MCksCiAgICAgICAgICAgICAgICAgICBtYWxlLmhvcnNlPWMoNiwzMCwyMCksIGZlbWFsZS5ob3JzZT1jKDYsMjgsMTkpKQpgYGAKCmBgYHtyfQpwZXRzMiA8LSBjYmluZChwZXRzWywnY291bnR5J10sIHN0YWNrKHBldHNbLC0xXSkpCm5hbWVzKHBldHMyKSA8LSBjKCdjb3VudHknLCdjb3VudCcsJ2luZCcpCmBgYAoKV2F0Y2ggb3V0IGZvciBmYWN0b3IgdmFyaWFibGVzLgoKYGBge3J9CnRyeUNhdGNoKHN0cnNwbGl0KHBldHMyWywnaW5kJ10sICdcXC4nKSwgZXJyb3I9ZnVuY3Rpb24oZSl7IGUgfSkKYGBgCgpgYGB7cn0KZ2VuZGVyQW5pbWFsIDwtIHN0cnNwbGl0KGFzLmNoYXJhY3RlcihwZXRzMlssJ2luZCddKSwgJ1xcLicpCnBldHMyWyxjKCdnZW5kZXInLCdhbmltYWwnKV0gPC0gTkEKZm9yKGkgaW4gc2VxKG5yb3cocGV0czIpKSkgewogIHBldHMyW2ksIGMoJ2dlbmRlcicsJ2FuaW1hbCcpXSA8LSBnZW5kZXJBbmltYWxbW2ldXQp9CnBldHMyWywnaW5kJ10gPC0gTlVMTApwZXRzMgpgYGAKCiMjIFZhcmlhYmxlcyBpbiBhIHJvdwoKYGBge3J9CnNldC5zZWVkKDEwMDApCmQxIDwtIHJub3JtKDEwMDAsIDUsIDIpCmQyIDwtIHJ1bmlmKDEwMDAsIDAsIDEwKQpkMyA8LSBycG9pcygxMDAwLCA1KQpkNCA8LSByYmlub20oMTAwMCwgMTAsIDAuNSkKeCA8LSBkYXRhLmZyYW1lKGRpc3RyaWJ1dGlvbj1jKHJlcCgnbm9ybWFsJywyKSxyZXAoJ3VuaWZvcm0nLDIpLHJlcCgncG9pc3NvbicsMikscmVwKCdiaW5vbWlhbCcsMikpLAogICAgICAgICAgIHN0YXQ9cmVwKGMoJ21lYW4nLCdzZCcpLDQpLAogICAgICAgICAgIHZhbHVlPWMobWVhbihkMSksIHNkKGQxKSwgbWVhbihkMiksIHNkKGQyKSwgbWVhbihkMyksIHNkKGQzKSwgbWVhbihkNCksIHNkKGQ0KSkpCmBgYAoKYGBge3J9CmNiaW5kKGRpc3RyaWJ1dGlvbj14W2MoMSwzLDUsNyksJ2Rpc3RyaWJ1dGlvbiddLCB1bnN0YWNrKHgsIGZvcm09dmFsdWV+c3RhdCkpCmBgYAoKIyMgTm9ybWFsaXphdGlvbiBhbmQgbWVyZ2luZwoKYGBge3J9CmVtcGxveWVlcyA8LSBkYXRhLmZyYW1lKGlkPTE6NCwgTmFtZT1jKCdFZGRpZScsJ0FuZHJlYScsJ1N0ZXZlJywnVGhlcmVzYScpLAogICAgICAgICAgICAgICAgICAgICAgICBqb2I9YygnZW5naW5lZXInLCdhY2NvdW50YW50Jywnc3RhdGlzdGljaWFuJywndGVjaG5pY2lhbicpKQpzZXQuc2VlZCgxMCkKaG91cnMgPC0gZGF0YS5mcmFtZShpZD1zYW1wbGUoNCwgMTAsIHJlcGxhY2U9VFJVRSksCiAgICAgICAgICAgICAgICAgICAgd2Vlaz0xOjEwLCBob3Vycz1zYW1wbGUoMzA6NjAsIDEwLCByZXBsYWNlPVRSVUUpKQptZXJnZShlbXBsb3llZXMsIGhvdXJzKQojIG1lcmdlKGVtcGxveWVlcywgaG91cnMsIGJ5Lng9J2lkJywgYnkueT0naWQnKQpgYGAKCmBgYHtyfQplbXBIb3VycyA8LSBtZXJnZShlbXBsb3llZXMsIGhvdXJzLCBhbGwueD1UUlVFKQplbXBIb3VycwpgYGAKCmBgYHtyfQp1bmlxdWUoZW1wSG91cnNbLGMoJ2lkJywnTmFtZScsJ2pvYicpXSkKZW1wSG91cnNbLGMoJ2lkJywnd2VlaycsJ2hvdXJzJyldCmBgYAoKRXhjbHVkZSByb3dzIHdpdGggbWlzc2luZyBkYXRhLgoKYGBge3J9CmVtcEhvdXJzWyFpcy5uYShlbXBIb3Vyc1ssJ3dlZWsnXSksIGMoJ2lkJywnd2VlaycsJ2hvdXJzJyldCmBgYAoKIyMgQ29tYmluaW5nIGRhdGEKCmBgYHtyfQpyYmluZChnZW5kZXJBbmltYWxbWzFdXSwgZ2VuZGVyQW5pbWFsW1syXV0sIGdlbmRlckFuaW1hbFtbM11dKQpgYGAKCmBgYHtyfQpkby5jYWxsKHJiaW5kLCBnZW5kZXJBbmltYWwpCmBgYAoKIyBTaW11bGF0aW9uCgpQbG90IG1lYW4gd2hpbGUgc2FtcGxpbmcgZnJvbSBub3JtYWwgZGlzdHJpYnV0aW9uCgpgYGB7cn0KbiA8LSAxMDAKaW1lYW4gPC0gbnVtZXJpYyhuKQpzbWVhbiA8LSBudW1lcmljKG4pCmZvcihpIGluIHNlcShuKSkgewogIGltZWFuW2ldIDwtIG1lYW4ocm5vcm0oMTApKQogIHNtZWFuW2ldIDwtIG1lYW4oaW1lYW5bc2VxKGkpXSkKfQpwbG90KGltZWFuLCB5bGFiPSdTYW1wbGUgTWVhbicpCmFibGluZShoPTAsIGx0eT0zKQpsaW5lcyhzbWVhbikKYGBgCgpgYGB7cn0KbiA8LSAxMDAwCnNpZyA8LSBudW1lcmljKG4pCmZvcihpIGluIHNlcShuKSkgewogIGdycDEgPC0gcm5vcm0oMTUsIDYwLCA1KQogIGdycDIgPC0gcm5vcm0oMTUsIDY1LCA1KQogIHNpZ1tpXSA8LSB0LnRlc3QoZ3JwMSwgZ3JwMiwgdmFyLmVxdWFsID0gVFJVRSkkcC52YWx1ZSA8IDAuMDUKfQptZWFuKHNpZykKYGBgCgpBZGQgc29tZSBmbGV4aWJpbGl0eSB0byBvdXIgc2ltdWxhdGlvbiBieSBjcmVhdGluZyBhIGZ1bmN0aW9uLgoKYGBge3J9CnRUZXN0UG93ZXIgPC0gZnVuY3Rpb24obj0xNSwgbXUxPTYwLCBtdTI9NjUsIHNkPTUsIG5zaW09MTAwMCkgewogIHNpZyA8LSBudW1lcmljKG5zaW0pCiAgZm9yKGkgaW4gc2VxKG5zaW0pKSB7CiAgICBncnAxIDwtIHJub3JtKG4sIG11MSwgc2QpCiAgICBncnAyIDwtIHJub3JtKG4sIG11Miwgc2QpCiAgICBzaWdbaV0gPC0gdC50ZXN0KGdycDEsIGdycDIsIHZhci5lcXVhbCA9IFRSVUUpJHAudmFsdWUgPCAwLjA1CiAgfQogIG1lYW4oc2lnKQp9CnRUZXN0UG93ZXIoKQp0VGVzdFBvd2VyKDI1LCBuc2ltPTEwMDAwKQpgYGAKClRoZXJlJ3MgYWxyZWFkeSBhIGZ1bmN0aW9uIHRvIGNvbXB1dGUgdGhlIHBvd2VyIG9mIGEgdHdvLXNhbXBsZSB0IHRlc3QuCgpgYGB7cn0KcG93ZXIudC50ZXN0KG49MjUsIGRlbHRhPTUsIHNkPTUpJHBvd2VyCmBgYAoKIyMgQm9vdHN0cmFwIHNhbXBsZQoKYGBge3J9CnRydWVfbXUgPC0gMAp4IDwtIHJub3JtKDEwMCwgdHJ1ZV9tdSkKUiA8LSA5OTkKcmVzIDwtIG1hdHJpeChucm93PVIsIG5jb2w9NikKY29sbmFtZXMocmVzKSA8LSBjKCdtdScsJ3NlJywnbGInLCd1YicsJ2NvdmVyYWdlJywnYmlhcycpCmZvcihpIGluIHNlcShSKSkgewogIHIgPC0gc2FtcGxlKHgsIHJlcGxhY2U9VFJVRSkKICByZXNbaSwnbXUnXSA8LSBtZWFuKHIpCiAgcmVzW2ksJ3NlJ10gPC0gc2QocikgLyBzcXJ0KGxlbmd0aChyKSkKfQpyZXNbLCdsYiddIDwtIHJlc1ssJ211J10gKyBxbm9ybSgwLjAyNSkgKiByZXNbLCdzZSddCnJlc1ssJ3ViJ10gPC0gcmVzWywnbXUnXSArIHFub3JtKDAuOTc1KSAqIHJlc1ssJ3NlJ10KcmVzWywnY292ZXJhZ2UnXSA8LSByZXNbLCdsYiddIDwgdHJ1ZV9tdSAmIHJlc1ssJ3ViJ10gPiB0cnVlX211CnJlc1ssJ2JpYXMnXSA8LSByZXNbLCdtdSddIC0gdHJ1ZV9tdQpgYGAKCmBgYHtyfQpyZXNbMTo1LF0KdmFscyA8LSBjb2xNZWFucyhyZXMpCmJhc2ljQ0kgPC0gMiAqIG1lYW4oeCkgLSBxdWFudGlsZShyZXNbLCdtdSddLCBwcm9icz1jKDAuOTc1LCAwLjAyNSkpCm91dCA8LSBzcHJpbnRmKCJlbXBpcmljYWwgU0U6ICUuM2YKTVNFOiAlLjNmCmJhc2ljIGJvb3RzdHJhcCBjb25maWRlbmNlIGludGVydmFsOiAoJS4zZiwgJS4zZikKY292ZXJhZ2UgcHJvYmFiaWxpdHk6ICUuM2YKbWVhbiBiaWFzOiAlLjNmCnNhbXBsZSBtZWFuOiAlLjNmCiIsIHNkKHJlc1ssJ211J10pLCB2YWxzWydzZSddLCBiYXNpY0NJWzFdLCBiYXNpY0NJWzJdLCB2YWxzWydjb3ZlcmFnZSddLCB2YWxzWydiaWFzJ10sIG1lYW4oeCkpCmNhdChvdXQpCmBgYAoKIyBDb250cm9sIFN0cnVjdHVyZXMKCkNvbnRyb2wgc3RydWN0dXJlcyBhcmUgdXNlZCB0byBjaGFuZ2UgaWYgYW5kIHdoZW4gbGluZXMgb2YgY29kZSBpbiBhIHByb2dyYW0gYXJlIHJ1bi4gIFRoZSBjb250cm9sIGNvbWVzIGZyb20gY29uZGl0aW9ucyBiZWluZyB0cnVlIG9yIGZhbHNlLgoKIyMgUm9vdCBmaW5kaW5nIHdpdGggdGhlIE5ld3Rvbi1SYXBoc29uIGFsZ29yaXRobQoKVGhlIGBmb3JgIGxvb3AgaGFzIGFscmVhZHkgYmVlbiBpbnRyb2R1Y2VkLiAgVHdvIG90aGVyIGltcG9ydGFudCBjb250cm9sIHN0cnVjdHVyZXMgYXJlIGB3aGlsZWAgbG9vcHMgYW5kIGBpZmAgc3RhdGVtZW50cy4KCmBgYHtyfQpmIDwtIGZ1bmN0aW9uKHgpIHheMyArIHheMiAtIDMqeCAtIDMKZnAgPC0gZnVuY3Rpb24oeCkgMyp4XjIgKyAyKnggLSAzCnBsb3QoZiwgeGxpbT1jKC00LDQpKQpgYGAKCmBgYHtyfQp4IDwtIC0yCmkgPC0gMQp3aGlsZShhYnMoZih4KSkgPiAxZS04KSB7CiAgeCA8LSB4IC0gZih4KSAvIGZwKHgpCiAgaSA8LSBpICsgMQogIGlmKGkgPiAyMCkgewogICAgcHJpbnQoImRvZXMgbm90IGNvbnZlcmdlIikKICAgIGJyZWFrCiAgfQp9CngKZih4KQpgYGAKCiMgV29ydGggTWVudGlvbmluZwoKIyMgdGlkeXZlcnNlCgpXaWNraGFtJ3MgIlRpZHkgRGF0YSIgcGhpbG9zb3BoeSB3YXMgY29uY3VycmVudCB3aXRoIHRoZSBkZXZlbG9wbWVudCBvZiBhIHN1aXRlIG9mIFIgcGFja2FnZXMgdXNlZnVsIGZvciBkYXRhIG1hbmFnZW1lbnQuICBUaGlzIGluY2x1ZGVzIGBnZ3Bsb3QyYCBhbmQgYGhhdmVuYCBhcyB3ZWxsIGFzIG1hbnkgb3RoZXJzIHN1Y2ggYXMgYGRwbHlyYCwgYHN0cmluZ3JgLCBhbmQgYHRpZHlyYC4KCiMjIGRhdGEudGFibGUKCmBkYXRhLnRhYmxlYCBpcyBhbm90aGVyIGJlbmVmaWNpYWwgcGFja2FnZSBmb3IgZGF0YSBtYW5pcHVsYXRpb24uICBJdCBwcm92aWRlcyBzaW1pbGFyIGZ1bmN0aW9uYWxpdHkgdG8gdGhlIGBkYXRhLmZyYW1lYCwgYnV0IHdvcmtzIHdlbGwgd2l0aCBsYXJnZSBkYXRhIHNldHMuCgpbZGF0YXNldHNdOiBodHRwOi8vYmlvc3RhdC5tYy52YW5kZXJiaWx0LmVkdS93aWtpL01haW4vRGF0YVNldHMgIkRhdGFTZXRzIgoKW3RpZHldOiBodHRwOi8vdml0YS5oYWQuY28ubnovcGFwZXJzL3RpZHktZGF0YS5wZGYgIlRpZHkgRGF0YSIKCltiYnJdOiBodHRwOi8vYmlvc3RhdC5tYy52YW5kZXJiaWx0LmVkdS90bXAvYmJyLnBkZiAiQmlvc3RhdGlzdGljcyBmb3IgQmlvbWVkaWNhbCBSZXNlYXJjaCIK